00000800:00000200:0.1F:1769073185.262553:0:3647:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [000000009d9f600b] -> 12345-192.168.204.160@tcp (2) 00000800:00000200:3.1F:1769073185.269694:0:3639:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [000000009d9f600b] -> 12345-192.168.204.160@tcp (2) 00000800:00000200:3.1:1769073185.269702:0:3639:0:(socklnd.c:175:ksocknal_destroy_peer()) peer_ni 12345-192.168.204.160@tcp 00000000e0ec1597 deleted 00000800:00000010:3.1:1769073185.269705:0:3639:0:(socklnd.c:184:ksocknal_destroy_peer()) kfreed 'peer_ni': 184 at 00000000e0ec1597. 00000800:00000200:3.1:1769073185.269710:0:3639:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 1, ksnr_connected 2 ksnr_max_conns 1 00000800:00000200:3.1:1769073185.269716:0:3639:0:(socklnd_cb.c:748:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.160@tcp ip 192.168.204.160:1023 00000800:00000200:3.1:1769073185.269719:0:3639:0:(socklnd_cb.c:767:ksocknal_queue_tx_locked()) Packet 00000000f3d6353e type 2, nob 96 niov 1 nkiov 0 00000800:00000200:3.1:1769073185.269907:0:3638:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 1, ksnr_connected 2 ksnr_max_conns 1 00000800:00000200:3.1:1769073185.270433:0:3640:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [000000009d9f600b] -> 12345-192.168.204.160@tcp (4) 00000800:00000200:3.1:1769073185.270436:0:3640:0:(socklnd.c:175:ksocknal_destroy_peer()) peer_ni 12345-192.168.204.160@tcp 00000000e0ec1597 deleted 00000800:00000010:3.1:1769073185.270439:0:3640:0:(socklnd.c:184:ksocknal_destroy_peer()) kfreed 'peer_ni': 184 at 00000000e0ec1597. 00000800:00000200:3.1:1769073185.270441:0:3640:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 3, ksnr_connected a ksnr_max_conns 1 00000800:00000200:3.1:1769073185.270536:0:3638:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 2, ksnr_connected e ksnr_max_conns 1 00010000:00000010:1.1F:1769073185.397902:0:0:0:(ldlm_resource.c:1330:__ldlm_resource_free()) slab-freed '(res)': 248 at 000000007a423df0. 00010000:00000010:1.1:1769073185.397940:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 000000008f1fe3fd. 00010000:00000010:2.1F:1769073185.543023:0:1052:0:(ldlm_resource.c:1330:__ldlm_resource_free()) slab-freed '(res)': 248 at 000000001affc3fa. 00010000:00000010:2.1:1769073185.543036:0:1052:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 00000000650a1c8f. 00010000:00000010:2.1:1769073185.554936:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 00000000b4c68e6d. 00010000:00000010:2.1:1769073185.554951:0:0:0:(ldlm_resource.c:1330:__ldlm_resource_free()) slab-freed '(res)': 248 at 00000000efcf93e9. 00010000:00000010:2.1:1769073185.554955:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 00000000f961ee7f. 00080000:00000010:3.1:1769073185.605424:0:1031:0:(osd_handler.c:1617:osd_oxc_free()) kfreed 'oxe': 68 at 00000000727940bf. 00010000:00010000:0.0:1769074750.764261:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764276:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764286:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764300:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764306:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764312:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764318:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773dc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074750.764322:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764329:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764338:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074750.764341:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074750.764344:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764350:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773dc00/0x7fe2323adfd51d82 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074750.764381:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773d600/0x7fe2323adfd51d7b lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa09 expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.764394:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074750.764396:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074750.764399:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773d600/0x7fe2323adfd51d7b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa09 expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.764407:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773d600, rc 0) 00000100:00100000:0.0:1769074750.764498:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:62429:x1855007616582400:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 501us (2251us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074750.764505:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13168 00000100:00100000:0.0:1769074750.786057:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.798590:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.801602:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.827543:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616584576 00000100:00100000:0.0:1769074750.827548:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616584576 00010000:00010000:0.0:1769074750.827569:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd51ccc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2134016->2215935) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a999 expref: 22 pid: 8399 timeout: 1838 lvb_type: 0 00000100:00100000:0.0:1769074750.827604:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6593 00000100:00100000:0.0:1769074750.827611:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616584576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074750.827621:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074750.827640:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd51ccc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2134016->2215935) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a999 expref: 22 pid: 8399 timeout: 1838 lvb_type: 0 00010000:00010000:0.0:1769074750.827654:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd51ccc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2134016->2215935) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a999 expref: 22 pid: 8399 timeout: 1838 lvb_type: 0 00010000:00010000:0.0:1769074750.827689:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074750.827691:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074750.827697:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd51d89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3973119) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa10 expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.827707:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd51d89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3973119) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa10 expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.827736:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd51d89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3973119) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa10 expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074750.827746:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625016064:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074750.830517:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422540 going to sleep for 11 seconds 00000100:00100000:0.0:1769074750.844999:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625016064:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074750.845044:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074750.845101:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616584576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17490us (17982us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074750.845114:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6593 00000100:00100000:0.0:1769074750.857413:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616584704 00000100:00100000:0.0:1769074750.857433:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616584704 00010000:00010000:0.0:1769074750.857484:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd51d5f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4091904->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9fb expref: 32 pid: 28954 timeout: 1838 lvb_type: 0 00000100:00100000:0.0:1769074750.857573:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6594 00000100:00100000:0.0:1769074750.857589:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616584704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074750.857615:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074750.857644:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd51d5f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4091904->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9fb expref: 32 pid: 28954 timeout: 1838 lvb_type: 0 00010000:00010000:0.0:1769074750.857661:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd51d5f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4091904->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9fb expref: 32 pid: 28954 timeout: 1838 lvb_type: 0 00010000:00010000:0.0:1769074750.857705:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd51d5f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4091904->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9fb expref: 32 pid: 28954 timeout: 1838 lvb_type: 0 00010000:00010000:0.0:1769074750.857725:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074750.857728:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074750.857737:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd51db3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa1e expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.857750:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd51db3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa1e expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.857799:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd51db3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa1e expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074750.857902:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625016192:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074750.857998:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422540 going to sleep for 11 seconds 00000100:00100000:0.0:1769074750.864607:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625016192:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074750.864640:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074750.864714:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616584704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7130us (7541us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074750.864729:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6594 00000100:00100000:0.0:1769074750.876774:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8160 00000100:00100000:0.0:1769074750.876795:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616584960:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074750.877192:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616584960:12345-192.168.204.60@tcp:3:cat.0 Request processed in 399us (1663us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074750.877202:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8160 00002000:00100000:0.0:1769074750.877210:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x2bc:0x0] (3969024->3973119) req@ffff89b5475dbb80 x1855007616584960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:286/0 lens 488/4536 e 0 to 0 dl 1769074761 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074750.877220:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475dbb80 with x1855007616584960 ext(3969024->3973119) 00002000:00010000:0.0:1769074750.877224:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475dbb80 00000100:00100000:0.0:1769074750.886121:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616585088 00000100:00100000:0.0:1769074750.886134:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616585088 00002000:00100000:0.0:1769074750.886199:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b67ff0ce00 x1855007616585088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:286/0 lens 488/0 e 0 to 0 dl 1769074761 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074750.886211:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ce00 with x1855007616585088 ext(0->1048575) 00002000:00010000:0.0:1769074750.886218:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0ce00 00000100:00100000:0.0:1769074750.886251:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616585216 00000100:00100000:0.0:1769074750.886255:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616585216 00002000:00100000:0.0:1769074750.886273:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b67ff0f800 x1855007616585216/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:286/0 lens 488/0 e 0 to 0 dl 1769074761 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074750.886281:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0f800 with x1855007616585216 ext(1048576->2097151) 00002000:00010000:0.0:1769074750.886285:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0f800 00000100:00100000:0.0:1769074750.886316:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8161 00000100:00100000:0.0:1769074750.886326:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616585088:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074750.899577:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616585088:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13236us (13590us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074750.899593:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8161 00002000:00100000:0.0:1769074750.899608:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b67ff0ce00 x1855007616585088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:286/0 lens 488/440 e 0 to 0 dl 1769074761 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074750.899619:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ce00 with x1855007616585088 ext(0->1048575) 00002000:00010000:0.0:1769074750.899627:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0ce00 00000100:00100000:0.0:1769074750.906181:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.937794:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.951276:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.966033:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616585984 00000100:00100000:0.0:1769074750.966046:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616585984 00002000:00100000:0.0:1769074750.966106:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b549203b80 x1855007616585984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:286/0 lens 488/0 e 0 to 0 dl 1769074761 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074750.966119:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203b80 with x1855007616585984 ext(2097152->3145727) 00002000:00010000:0.0:1769074750.966127:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549203b80 00000100:00100000:0.0:1769074750.966176:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8163 00000100:00100000:0.0:1769074750.966185:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616585984:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074750.978376:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616585984:12345-192.168.204.60@tcp:3:cat.0 Request processed in 12177us (12443us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074750.978393:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8163 00002000:00100000:0.0:1769074750.978410:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b549203b80 x1855007616585984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:286/0 lens 488/440 e 0 to 0 dl 1769074761 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074750.978428:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203b80 with x1855007616585984 ext(2097152->3145727) 00002000:00010000:0.0:1769074750.978434:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549203b80 00000100:00100000:0.0:1769074750.983395:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074750.994870:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616586240 00000100:00100000:0.0:1769074750.994881:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616586240 00000100:00100000:0.0:1769074750.994951:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13422 00000100:00100000:0.0:1769074750.994962:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62432:x1855007616586240:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074750.994985:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074750.995007:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773ce00/0x7fe2323adfd51e07 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aa56 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.995051:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074750.995054:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074750.995063:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773ce00/0x7fe2323adfd51e07 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa56 expref: 30 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.995078:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773ce00/0x7fe2323adfd51e07 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa56 expref: 30 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074750.995090:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ce00, rc 0) 00000100:00100000:0.0:1769074750.995153:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:62432:x1855007616586240:12345-192.168.204.60@tcp:101:cat.0 Request processed in 192us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074750.995160:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13422 00000100:00100000:0.0:1769074751.008916:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625016448:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074751.008939:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.008982:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616586112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 25123us (25598us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.008991:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6595 00000100:00080000:0.0:1769074751.021780:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074751.048495:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616586624 00000100:00100000:0.0:1769074751.048515:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616586624 00002000:00100000:0.0:1769074751.048572:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d103b80 x1855007616586624/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.048584:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103b80 with x1855007616586624 ext(2097152->4194303) 00002000:00010000:0.0:1769074751.048590:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d103b80 00000100:00100000:0.0:1769074751.048615:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8166 00000100:00100000:0.0:1769074751.048623:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2407:x1855007616586624:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074751.098229:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2407:x1855007616586624:12345-192.168.204.60@tcp:3:cat.0 Request processed in 49594us (54846us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074751.098244:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8166 00002000:00100000:0.0:1769074751.098256:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d103b80 x1855007616586624/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/440 e 0 to 0 dl 1769074762 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.098266:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103b80 with x1855007616586624 ext(2097152->4194303) 00002000:00010000:0.0:1769074751.098272:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d103b80 00000100:00100000:0.0:1769074751.104565:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616586880 00000100:00100000:0.0:1769074751.104575:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616586880 00002000:00100000:0.0:1769074751.104636:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5475da680 x1855007616586880/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.104648:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475da680 with x1855007616586880 ext(4194304->4808703) 00002000:00010000:0.0:1769074751.104654:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475da680 00000100:00100000:0.0:1769074751.104742:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8168 00000100:00100000:0.0:1769074751.104752:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616586880:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074751.171568:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.200729:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.216503:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.236876:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.244770:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.245967:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616588800 00000100:00100000:0.0:1769074751.245973:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616588800 00000100:00100000:0.0:1769074751.246012:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4575 00000100:00100000:0.0:1769074751.246018:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62434:x1855007616588800:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074751.246251:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62434:x1855007616588800:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 232us (369us total) trans 8589943760 rc 0/0 00000100:00100000:0.0:1769074751.246259:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4575 00000100:00100000:0.0:1769074751.293731:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.299733:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616589568 00000100:00100000:0.0:1769074751.299743:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616589568 00002000:00100000:0.0:1769074751.299792:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1654784->1658879) req@ffff89b54cde7b80 x1855007616589568/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.299803:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7b80 with x1855007616589568 ext(1654784->1658879) 00002000:00010000:0.0:1769074751.299839:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde7b80 00000100:00100000:0.0:1769074751.299872:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8171 00000100:00100000:0.0:1769074751.299886:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616589568:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074751.300066:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616589568:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 183us (442us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074751.300083:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8171 00002000:00100000:0.0:1769074751.300094:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1654784->1658879) req@ffff89b54cde7b80 x1855007616589568/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/4536 e 0 to 0 dl 1769074762 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.300103:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7b80 with x1855007616589568 ext(1654784->1658879) 00002000:00010000:0.0:1769074751.300107:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde7b80 00000100:00100000:0.0:1769074751.318998:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13428 00000100:00100000:0.0:1769074751.319008:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cddf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616589824:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074751.319027:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074751.319039:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aa87 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.319052:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.319055:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.319065:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.319080:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.319092:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773c400, rc 0) 00000100:00100000:0.0:1769074751.319127:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cddf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616589824:12345-192.168.204.60@tcp:101:cat.0 Request processed in 121us (798us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.319133:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13428 00002000:00010000:0.0:1769074751.323799:0:36516:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074749 -> 1769074750 00002000:00010000:0.0:1769074751.323858:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074749 -> 1769074750 00002000:00010000:0.0:1769074751.323862:0:36516:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074749 -> 1769074750 00000100:00100000:0.0:1769074751.323894:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625017984:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074751.323949:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd51e3f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa80 expref: 22 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.323977:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd51e3f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa80 expref: 22 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.323992:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e800, rc 0) 00000100:00100000:0.0:1769074751.324083:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cdddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616589696:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5454us (5833us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074751.324096:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13427 00000100:00100000:0.0:1769074751.326575:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616590080 00000100:00100000:0.0:1769074751.326584:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616590080 00002000:00100000:0.0:1769074751.326631:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4804608->4808703) req@ffff89b54cddc380 x1855007616590080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.326643:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cddc380 with x1855007616590080 ext(4804608->4808703) 00002000:00010000:0.0:1769074751.326649:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cddc380 00000100:00100000:0.0:1769074751.326686:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8172 00000100:00100000:0.0:1769074751.326694:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cddc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616590080:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074751.326871:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cddc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616590080:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 176us (417us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074751.326878:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8172 00002000:00100000:0.0:1769074751.326884:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4804608->4808703) req@ffff89b54cddc380 x1855007616590080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/4536 e 0 to 0 dl 1769074762 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.326897:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cddc380 with x1855007616590080 ext(4804608->4808703) 00002000:00010000:0.0:1769074751.326900:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cddc380 00000100:00100000:0.0:1769074751.328751:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.339726:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616590208 00000100:00100000:0.0:1769074751.339737:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616590208 00010000:00010000:0.0:1769074751.339779:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd51d0b lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9bc expref: 18 pid: 8399 timeout: 1839 lvb_type: 1 00000100:00100000:0.0:1769074751.339859:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6604 00000100:00100000:0.0:1769074751.339869:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b682062450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616590208:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.339886:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.339911:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd51d0b lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9bc expref: 18 pid: 8399 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.339923:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd51d0b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9bc expref: 18 pid: 8399 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.339950:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.340003:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b682062450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616590208:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 134us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.340021:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6604 00000100:00100000:0.0:1769074751.350535:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616590336 00000100:00100000:0.0:1769074751.350547:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616590336 00010000:00010000:0.0:1769074751.350588:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeac00/0x7fe2323adfd51d51 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9ed expref: 15 pid: 36516 timeout: 1839 lvb_type: 1 00000100:00100000:0.0:1769074751.350636:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616590464 00000100:00100000:0.0:1769074751.350639:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616590464 00010000:00010000:0.0:1769074751.350653:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd51ca2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a98b expref: 33 pid: 8399 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.350693:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6605 00000100:00100000:0.0:1769074751.350702:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cdde300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616590336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.350718:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.350741:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeac00/0x7fe2323adfd51d51 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9ed expref: 15 pid: 36516 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.350754:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeac00/0x7fe2323adfd51d51 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9ed expref: 15 pid: 36516 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.350778:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeac00/0x7fe2323adfd51d51 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9ed expref: 15 pid: 36516 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.350800:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.350899:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cdde300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616590336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 197us (546us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.350907:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6605 00000100:00100000:0.0:1769074751.350923:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6606 00000100:00100000:0.0:1769074751.350930:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cddc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616590464:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.350937:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.350945:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd51ca2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a98b expref: 33 pid: 8399 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.350953:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd51ca2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a98b expref: 33 pid: 8399 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.350966:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd51ca2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a98b expref: 33 pid: 8399 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.350975:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.350977:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.350983:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 32 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.350993:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 32 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.351712:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 32 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074751.351732:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625018496:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074751.351944:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a4229c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.356902:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.358949:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625018496:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074751.358976:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.359039:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cddc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616590464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8108us (8625us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.359054:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6606 00000100:00100000:0.0:1769074751.365417:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616590848 00000100:00100000:0.0:1769074751.365426:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616590848 00002000:00100000:0.0:1769074751.365470:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1220608->1224703) req@ffff89b54cddca80 x1855007616590848/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.365488:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cddca80 with x1855007616590848 ext(1220608->1224703) 00002000:00010000:0.0:1769074751.365494:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cddca80 00000100:00100000:0.0:1769074751.365517:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8173 00000100:00100000:0.0:1769074751.365525:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cddca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616590848:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074751.365679:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cddca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616590848:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 154us (343us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074751.365686:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8173 00002000:00100000:0.0:1769074751.365692:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1220608->1224703) req@ffff89b54cddca80 x1855007616590848/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/4536 e 0 to 0 dl 1769074762 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074751.365699:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cddca80 with x1855007616590848 ext(1220608->1224703) 00002000:00010000:0.0:1769074751.365703:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cddca80 00000100:00100000:0.0:1769074751.393041:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616590976 00000100:00100000:0.0:1769074751.393051:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616590976 00010000:00010000:0.0:1769074751.393089:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1654784->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa79 expref: 31 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.393134:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6607 00000100:00100000:0.0:1769074751.393143:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cddea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616590976:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.393158:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.393179:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1654784->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa79 expref: 31 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.393193:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1654784->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa79 expref: 31 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.393216:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1654784->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa79 expref: 31 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.393233:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.393235:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.393242:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.393252:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.393303:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074751.393319:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625018752:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074751.393412:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a4229c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.396415:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625018752:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074751.396456:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.396532:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cddea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616590976:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3388us (3625us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.396542:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6607 00000100:00100000:0.0:1769074751.424158:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616591232 00000100:00100000:0.0:1769074751.424169:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616591232 00000100:00100000:0.0:1769074751.439859:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616591488 00000100:00100000:0.0:1769074751.439875:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616591488 00010000:00010000:0.0:1769074751.439917:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 34 pid: 36516 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.439960:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6608 00000100:00100000:0.0:1769074751.439970:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cdddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616591488:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.439984:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1769074751.440002:0:8424:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074749 -> 1769074751 00002000:00010000:0.0:1769074751.440005:0:8424:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074749 -> 1769074751 00010000:00010000:0.0:1769074751.440021:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 34 pid: 36516 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.440036:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 34 pid: 36516 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.440061:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd51e4d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1224703) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa8e expref: 34 pid: 36516 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.440079:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.440081:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.440089:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.440099:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.440141:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074751.440157:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625019008:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074751.440243:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a4229c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.446629:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.476210:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.534907:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.543916:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.621991:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.647502:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.650896:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.665792:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.665870:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.665975:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000004:00200000:1.0:1769074751.712379:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:1.0:1769074751.712385:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074751.712407:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51ee0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712419:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51ee0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712432:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712439:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712444:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712451:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712456:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910dc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074751.712461:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712469:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712477:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074751.712480:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074751.712485:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712491:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ed2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712501:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712506:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712511:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712515:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910da00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074751.712518:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712524:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712530:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294700) 00010000:00010000:1.0:1769074751.712533:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074751.712535:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712540:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51ed9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074751.712574:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ecb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aac6 expref: 19 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.712586:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51ee0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aac6 expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.712592:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074751.712593:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074751.712596:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:1.0:1769074751.712628:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:62454:x1855007616593664:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 18679us (19260us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.712636:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13175 00000100:00100000:1.0:1769074751.747964:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616594048 00000100:00100000:1.0:1769074751.747977:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616594048 00002000:00100000:1.0:1769074751.748050:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5436d6a00 x1855007616594048/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.748062:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6a00 with x1855007616594048 ext(1048576->2097151) 00002000:00010000:1.0:1769074751.748069:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d6a00 00000100:00100000:1.0:1769074751.748109:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8178 00000100:00100000:1.0:1769074751.748119:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616594048:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074751.786997:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616594176 00000100:00100000:1.0:1769074751.787027:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616594176 00000100:00100000:1.0:1769074751.787102:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616594304 00000100:00100000:1.0:1769074751.787106:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616594304 00000100:00100000:1.0:1769074751.787130:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616594432 00000100:00100000:1.0:1769074751.787133:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616594432 00000100:00100000:1.0:1769074751.787163:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13434 00000100:00100000:1.0:1769074751.787174:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616594176:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074751.787193:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074751.787214:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aacd expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.787238:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.787240:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.787250:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.787267:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.787281:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:1.0:1769074751.787345:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616594176:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 172us (15540us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.787352:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13434 00000100:00100000:1.0:1769074751.787373:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13435 00000100:00100000:1.0:1769074751.787378:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616594304:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074751.787385:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074751.787393:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aad4 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.787403:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.787405:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.787410:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.787419:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.787428:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:1.0:1769074751.787446:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616594304:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 67us (15520us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.787452:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13435 00000100:00100000:1.0:1769074751.787463:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13436 00000100:00100000:1.0:1769074751.787468:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2409:x1855007616594432:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074751.787617:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074751.787629:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ef5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aadb expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.787644:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.787647:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.787652:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ef5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aadb expref: 15 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.787674:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd51ef5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aadb expref: 15 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.787683:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910dc00, rc 0) 00000100:00100000:1.0:1769074751.787705:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616594432:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 236us (15635us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.787710:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13436 00000100:00100000:1.0:1769074751.793212:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616594048:12345-192.168.204.60@tcp:3:cat.0 Request processed in 44678us (44991us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074751.793229:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8178 00002000:00100000:1.0:1769074751.793244:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5436d6a00 x1855007616594048/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/440 e 0 to 0 dl 1769074762 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.793257:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6a00 with x1855007616594048 ext(1048576->2097151) 00002000:00010000:1.0:1769074751.793263:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d6a00 00000100:00100000:0.0:1769074751.807201:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074751.807289:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616594944 00000100:00100000:1.0:1769074751.807300:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616594944 00000100:00100000:1.0:1769074751.807429:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13439 00000100:00100000:1.0:1769074751.807440:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62447:x1855007616594944:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074751.807465:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074751.807493:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aaf0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.807526:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd51e31 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa72 expref: 35 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.807536:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b08e800/0x7fe2323adfd51efc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae2 expref: 15 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.807544:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.807551:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.807553:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.807557:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3698688->4194303] (req 3698688->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.807592:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.807605:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 1839 lvb_type: 1 00000100:00100000:1.0:1769074751.807620:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625019392:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074751.807729:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b08e800/0x7fe2323adfd51efc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae2 expref: 15 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074751.807739:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b08e800/0x7fe2323adfd51efc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae2 expref: 15 pid: 8399 timeout: 1839 lvb_type: 1 00000100:00100000:1.0:1769074751.807748:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625019520:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074751.807777:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd51e31 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa72 expref: 35 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.807786:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd51e31 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa72 expref: 35 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:1.0:1769074751.807795:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625019648:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074751.807862:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.817429:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074751.835245:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625019392:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074751.835267:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625019520:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074751.835279:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625019648:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074751.835318:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3698688->4194303] (req 3698688->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.835343:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:1.0:1769074751.835492:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62447:x1855007616594944:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 28052us (28304us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.835502:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13439 00000100:00100000:0.0:1769074751.836077:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616595456 00000100:00100000:0.0:1769074751.836088:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616595456 00010000:00010000:0.0:1769074751.836129:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd51e31 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa72 expref: 34 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.836207:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616595584 00000100:00100000:0.0:1769074751.836211:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616595584 00010000:00010000:0.0:1769074751.836226:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 1839 lvb_type: 1 00000100:00100000:1.0:1769074751.837165:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6612 00000100:00100000:1.0:1769074751.837176:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616595584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074751.837187:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074751.837200:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 1839 lvb_type: 1 00010000:00010000:1.0:1769074751.837213:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 1839 lvb_type: 1 00010000:00010000:1.0:1769074751.837227:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773c400/0x7fe2323adfd51e46 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa87 expref: 19 pid: 28954 timeout: 1839 lvb_type: 1 00010000:00010000:1.0:1769074751.837238:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.837379:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6613 00000100:00100000:0.0:1769074751.837390:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616595328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.837399:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.837409:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b08e800/0x7fe2323adfd51efc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae2 expref: 16 pid: 8399 timeout: 1839 lvb_type: 1 00000100:00100000:1.0:1769074751.837777:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616595584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 602us (2702us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.837791:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6612 00010000:00010000:0.0:1769074751.838075:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b08e800/0x7fe2323adfd51efc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae2 expref: 16 pid: 8399 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.838093:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b08e800/0x7fe2323adfd51efc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae2 expref: 16 pid: 8399 timeout: 1839 lvb_type: 1 00010000:00010000:0.0:1769074751.838105:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.838107:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.838116:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 17 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.838127:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 17 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.838168:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 17 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074751.838186:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625019904:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074751.838740:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.839187:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616595840 00000100:00100000:0.0:1769074751.839196:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616595840 00010000:00010000:0.0:1769074751.839225:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51ee0 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aac6 expref: 20 pid: 6512 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.839258:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6614 00000100:00100000:0.0:1769074751.839377:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+20:2407:x1855007616595840:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.839394:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.839402:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51ee0 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aac6 expref: 20 pid: 6512 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.839861:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51ee0 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aac6 expref: 20 pid: 6512 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.839886:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074751.839892:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd51f18 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074751.840185:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd51f18 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074751.840196:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd51f18 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074751.840218:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.840402:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+20:2407:x1855007616595840:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1133us (1337us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.840411:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6614 00000100:00100000:1.0:1769074751.840688:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616595712 00000100:00100000:1.0:1769074751.840697:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616595712 00000100:00100000:0.0:1769074751.840841:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13440 00000100:00100000:0.0:1769074751.840854:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62453:x1855007616595712:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074751.840873:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074751.840890:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aafe expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.840928:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.840937:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 12 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.840946:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773ce00/0x7fe2323adfd51e07 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa56 expref: 32 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.840952:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.840954:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.840958:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2805760->3145727] (req 2805760->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 32 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.840986:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773ce00/0x7fe2323adfd51e07 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa56 expref: 32 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.841000:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54773ce00/0x7fe2323adfd51e07 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa56 expref: 32 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.841013:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625020032:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074751.841079:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 12 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.841096:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 12 pid: 8399 timeout: 1839 lvb_type: 1 00000100:00100000:0.0:1769074751.841106:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625020160:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074751.841135:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.841145:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 21 pid: 36516 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.841154:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625020288:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074751.841176:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074751.853589:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616596096 00000100:00100000:1.0:1769074751.853595:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616596096 00000100:00100000:1.0:1769074751.856247:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13441 00000100:00100000:1.0:1769074751.856276:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:62452:x1855007616596096:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074751.856297:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074751.856319:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab05 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.856358:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 17 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.856368:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074751.856370:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074751.856374:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 34 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.856411:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 17 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.856423:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 17 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:1.0:1769074751.856436:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625020416:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074751.857364:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625019904:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074751.857384:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.857442:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616595328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 20053us (22287us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.857459:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6613 00000100:00100000:1.0:1769074751.857798:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074751.864038:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074751.864069:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616596480 00000100:00100000:1.0:1769074751.864077:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616596480 00010000:00010000:1.0:1769074751.864105:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 13 pid: 8399 timeout: 1839 lvb_type: 1 00000100:00100000:1.0:1769074751.864138:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6616 00000100:00100000:1.0:1769074751.864144:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616596480:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074751.864156:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074751.864178:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 13 pid: 8399 timeout: 1839 lvb_type: 1 00010000:00010000:1.0:1769074751.864193:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 13 pid: 8399 timeout: 1839 lvb_type: 1 00010000:00010000:1.0:1769074751.864216:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074751.864219:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd51f03 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aae9 expref: 13 pid: 8399 timeout: 1839 lvb_type: 1 00000100:00100000:0.0:1769074751.864236:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625020160:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074751.864249:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625020288:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074751.864262:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616596480:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 117us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.864270:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6616 00010000:00010000:0.0:1769074751.864297:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2805760->3145727] (req 2805760->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 31 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.864312:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924600, rc 0) 00000100:00100000:0.0:1769074751.864343:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62453:x1855007616595712:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 23492us (25335us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.864351:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13440 00000100:00100000:1.0:1769074751.873352:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616596608 00000100:00100000:1.0:1769074751.873372:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616596608 00010000:00010000:1.0:1769074751.873433:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 22 pid: 36516 timeout: 1839 lvb_type: 0 00000100:00100000:1.0:1769074751.873488:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6617 00000100:00100000:1.0:1769074751.873498:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b643b0c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616596608:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074751.873511:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074751.873532:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 22 pid: 36516 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.873547:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 22 pid: 36516 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.873569:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b546f91600/0x7fe2323adfd51df9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa48 expref: 22 pid: 36516 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.873587:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.873590:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.873598:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.873610:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.873650:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074751.873676:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625020672:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074751.873731:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397f00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074751.873749:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b647316240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074751.876218:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074751.876265:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.876572:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616596864 00000100:00100000:0.0:1769074751.876577:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616596864 00000100:00100000:0.0:1769074751.876668:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13442 00000100:00100000:0.0:1769074751.876678:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62447:x1855007616596864:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074751.876688:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074751.876705:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd51f42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab13 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.876729:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.876736:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074751.876738:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074751.876742:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd51f42 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->3358719] (req 3145728->3358719) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab13 expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.876777:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.876789:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 30 pid: 28954 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.876831:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625020800:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074751.876936:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422540 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.883546:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625020800:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074751.883583:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd51f42 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->3358719] (req 3145728->3358719) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab13 expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.883605:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b08e800, rc 0) 00000100:00100000:0.0:1769074751.883684:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62447:x1855007616596864:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 7007us (7418us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.883693:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13442 00000100:00100000:1.0:1769074751.884743:0:24026:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616596992 00000100:00100000:1.0:1769074751.884753:0:24026:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616596992 00002000:00100000:1.0:1769074751.884796:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2bc:0x0] (2805760->2809855) req@ffff89b67a0f6680 x1855007616596992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.884803:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6680 with x1855007616596992 ext(2805760->2809855) 00010000:00010000:1.0:1769074751.884836:0:24026:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 31 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.884849:0:24026:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 31 pid: 28954 timeout: 1839 lvb_type: 0 00002000:00010000:1.0:1769074751.884856:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a0f6680 00000100:00100000:1.0:1769074751.884885:0:24026:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 651 00000100:00100000:1.0:1769074751.884905:0:24026:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616596992:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074751.885022:0:24026:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616596992:12345-192.168.204.60@tcp:3:cat.0 Request processed in 130us (1593us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074751.885027:0:24026:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 651 00002000:00100000:1.0:1769074751.885031:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2bc:0x0] (2805760->2809855) req@ffff89b67a0f6680 x1855007616596992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/4536 e 0 to 0 dl 1769074762 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.885036:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6680 with x1855007616596992 ext(2805760->2809855) 00010000:00010000:1.0:1769074751.885040:0:24026:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 31 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.885046:0:24026:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 31 pid: 28954 timeout: 1839 lvb_type: 0 00002000:00010000:1.0:1769074751.885049:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a0f6680 00000100:00100000:1.0:1769074751.891228:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616596736:12345-192.168.204.60@tcp:4:cat.0 Request processed in 14797us (15006us total) trans 17179874407 rc 0/0 00000100:00100000:1.0:1769074751.891244:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 650 00002000:00100000:1.0:1769074751.891259:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3698688->4194303) req@ffff89b5475d8380 x1855007616596736/t17179874407(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:287/0 lens 488/448 e 0 to 0 dl 1769074762 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.891270:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8380 with x1855007616596736 ext(3698688->4194303) 00010000:00010000:1.0:1769074751.891280:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 20 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.891295:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 20 pid: 27843 timeout: 1839 lvb_type: 0 00002000:00010000:1.0:1769074751.891305:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5475d8380 00000100:00100000:1.0:1769074751.891365:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625020544:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074751.891393:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.893651:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.893671:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074751.893723:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:1.0:1769074751.893727:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074751.893744:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.893768:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd51f34 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab0c expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.893779:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab0c expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.893784:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074751.893786:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074751.893788:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b7200, rc 0) 00000100:00100000:1.0:1769074751.893860:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:62454:x1855007616596352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 33024us (37002us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.893867:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13177 00000100:00100000:1.0:1769074751.895435:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616597248 00000100:00100000:1.0:1769074751.895446:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616597248 00010000:00010000:1.0:1769074751.895485:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 20 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:1.0:1769074751.895530:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6619 00000100:00100000:1.0:1769074751.895539:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616597248:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074751.895551:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074751.895568:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074750 -> 1769074751 00002000:00010000:1.0:1769074751.895571:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074750 -> 1769074751 00010000:00010000:1.0:1769074751.895577:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 20 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.895588:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 20 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.895610:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f0a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3698688->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaf0 expref: 20 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.895625:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.895627:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.895635:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 33 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.895644:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 33 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.895691:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 33 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074751.895709:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625020928:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074751.895794:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397240 going to sleep for 11 seconds 00000001:00080000:0.0:1769074751.902851:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841089 is committed 00000001:00080000:0.0:1769074751.902875:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841087 is committed 00000001:00080000:0.0:1769074751.902930:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841076 is committed 00000001:00080000:0.0:1769074751.902964:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841052 is committed 00000100:00100000:1.0:1769074751.903872:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616597376 00000100:00100000:1.0:1769074751.903882:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616597376 00002000:00100000:1.0:1769074751.903943:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2805760->3145727) req@ffff89b67fb97480 x1855007616597376/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/0 e 0 to 0 dl 1769074762 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.903953:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb97480 with x1855007616597376 ext(2805760->3145727) 00010000:00010000:1.0:1769074751.903964:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 17 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.903979:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 17 pid: 28954 timeout: 1839 lvb_type: 0 00002000:00010000:1.0:1769074751.903987:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fb97480 00000100:00100000:1.0:1769074751.904021:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 652 00000100:00100000:1.0:1769074751.904029:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616597376:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074751.905962:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.915850:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.915942:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625021056:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074751.915980:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3416063] (req 3145728->3416063) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.916003:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b719800, rc 0) 00000100:00100000:0.0:1769074751.916057:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62453:x1855007616597504:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 9865us (10108us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.916067:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13443 00000100:00100000:0.0:1769074751.916100:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616597760 00000100:00100000:0.0:1769074751.916106:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616597760 00000100:00100000:0.0:1769074751.916149:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616597888 00000100:00100000:0.0:1769074751.916152:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616597888 00000100:00100000:0.0:1769074751.916175:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13444 00000100:00100000:0.0:1769074751.916184:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616597760:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074751.916199:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074751.916220:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd51f50 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab21 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.916242:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.916257:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd51f50 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab21 expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.916282:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd51f50 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab21 expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.916295:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4000, rc 0) 00000100:00100000:0.0:1769074751.916326:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616597760:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 142us (348us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074751.916333:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13444 00000100:00100000:0.0:1769074751.916349:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13445 00000100:00100000:0.0:1769074751.916356:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616597888:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074751.916363:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074751.916372:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd51f57 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab28 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074751.916414:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625021184:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074751.916471:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074751.916690:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625020928:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074751.916702:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074751.916727:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2406:x1855007616597248:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 21190us (21540us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074751.916735:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6619 00000100:00100000:1.0:1769074751.927949:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616597376:12345-192.168.204.60@tcp:4:cat.0 Request processed in 23909us (24184us total) trans 21474841090 rc 0/0 00000100:00100000:1.0:1769074751.927967:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 652 00002000:00100000:1.0:1769074751.927981:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2805760->3145727) req@ffff89b67fb97480 x1855007616597376/t21474841090(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:287/0 lens 488/448 e 0 to 0 dl 1769074762 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074751.927991:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb97480 with x1855007616597376 ext(2805760->3145727) 00010000:00010000:1.0:1769074751.928003:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 19 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.928019:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 19 pid: 28954 timeout: 1839 lvb_type: 0 00002000:00010000:1.0:1769074751.928027:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fb97480 00002000:00010000:0.0:1769074751.930607:0:27843:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074750 -> 1769074751 00002000:00010000:0.0:1769074751.930618:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074750 -> 1769074751 00002000:00010000:0.0:1769074751.930620:0:27843:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074750 -> 1769074751 00000100:00100000:0.0:1769074751.930643:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625021184:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074751.930713:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd51f57 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab28 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.930738:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd51f57 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab28 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074751.930751:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea5400, rc 0) 00000100:00100000:0.0:1769074751.930856:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616597888:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 14499us (14810us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074751.930866:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13445 00000100:00100000:1.0:1769074751.940339:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616598016 00000100:00100000:1.0:1769074751.940351:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616598016 00010000:00010000:1.0:1769074751.940391:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 19 pid: 28954 timeout: 1839 lvb_type: 0 00000100:00100000:1.0:1769074751.940434:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6620 00000100:00100000:1.0:1769074751.940443:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616598016:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074751.940468:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074751.940491:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 19 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.940506:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 19 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.940546:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b679924600/0x7fe2323adfd51f1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2805760->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aafe expref: 19 pid: 28954 timeout: 1839 lvb_type: 0 00010000:00010000:1.0:1769074751.940563:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.940565:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074751.940573:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd51f42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3358719) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab13 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.940583:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd51f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3358719) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab13 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.940621:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd51f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3358719) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab13 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074751.940636:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625021312:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074751.940732:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397240 going to sleep for 11 seconds 00000100:00100000:0.0:1769074751.947418:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074751.947537:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616598400 00000100:00100000:1.0:1769074751.947547:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616598400 00000100:00100000:1.0:1769074751.947609:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13178 00000100:00100000:1.0:1769074751.947619:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:62454:x1855007616598400:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074751.947641:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074751.947673:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f5e lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab2f expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.947784:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947803:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947846:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947860:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947866:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947873:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947880:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910ce00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074751.947884:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.947891:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074751.949601:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625021312:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074751.949632:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.949713:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616598016:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9270us (9470us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.949725:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6620 00010000:00010000:1.0:1769074751.950376:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074751.950381:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074751.950388:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.950396:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd51f65 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 14 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074751.950449:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f5e lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 13 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab2f expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.950465:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074751.950467:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074751.950471:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd51f5e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 13 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab2f expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074751.950480:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:0.0:1769074751.958644:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:62454:x1855007616598400:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11017us (11227us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074751.958667:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13178 00000100:00100000:1.0:1769074751.970169:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616598912 00000100:00100000:1.0:1769074751.970187:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616598912 00000100:00100000:1.0:1769074751.970267:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4578 00000100:00100000:1.0:1769074751.970280:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62447:x1855007616598912:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074751.970662:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62447:x1855007616598912:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 376us (3159us total) trans 8589943767 rc 0/0 00000100:00100000:0.0:1769074751.970672:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4578 00000100:00100000:0.0:1769074751.972751:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616599040 00000100:00100000:0.0:1769074751.972763:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616599040 00010000:00010000:0.0:1769074751.972860:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 21 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074751.972898:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6621 00000100:00100000:0.0:1769074751.972907:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616599040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074751.972920:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074751.972941:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 21 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.972954:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 21 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.972979:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd51f2d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab05 expref: 21 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074751.972995:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.973007:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074751.973016:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.973026:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074751.973069:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074751.973082:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625021568:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074751.973166:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a422540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074751.973734:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.974846:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616599424 00000100:00100000:0.0:1769074751.974860:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616599424 00000100:00100000:0.0:1769074751.974921:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4579 00000100:00100000:0.0:1769074751.974930:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b68164fc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:62454:x1855007616599424:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074751.975490:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b68164fc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+20:62454:x1855007616599424:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 559us (1755us total) trans 8589943768 rc 0/0 00000100:00100000:0.0:1769074751.975499:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4579 00000100:00100000:0.0:1769074751.993039:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074751.994750:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625021568:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074751.994775:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074751.994861:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616599040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 21954us (23867us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074751.994872:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6621 00000100:00100000:0.0:1769074752.004467:0:24026:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616599680 00000100:00100000:0.0:1769074752.004480:0:24026:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616599680 00002000:00100000:0.0:1769074752.004652:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3411968->3416063) req@ffff89b67a0f4000 x1855007616599680/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.004682:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f4000 with x1855007616599680 ext(3411968->3416063) 00002000:00010000:0.0:1769074752.004690:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f4000 00000100:00100000:0.0:1769074752.004731:0:24026:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616599808 00000100:00100000:0.0:1769074752.004736:0:24026:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616599808 00002000:00100000:0.0:1769074752.004771:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b67a0f6300 x1855007616599808/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.004782:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6300 with x1855007616599808 ext(2097152->3145727) 00002000:00010000:0.0:1769074752.004789:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f6300 00000100:00080000:0.0:1769074752.005982:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074752.016201:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616600064 00000100:00100000:0.0:1769074752.016214:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616600064 00000100:00100000:0.0:1769074752.016291:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4580 00000100:00100000:0.0:1769074752.016302:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62453:x1855007616600064:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074752.016595:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62453:x1855007616600064:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 293us (1747us total) trans 8589943769 rc 0/0 00000100:00100000:0.0:1769074752.016603:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4580 00000100:00100000:1.0:1769074752.049335:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.049378:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.081032:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:0.0:1769074752.081964:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074752.083240:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616600576 00000100:00100000:0.0:1769074752.083248:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616600576 00000100:00100000:0.0:1769074752.083295:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13448 00000100:00100000:0.0:1769074752.083303:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616600576:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074752.083317:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.083332:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab44 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.083350:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.083352:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.083362:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 12 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.083381:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 12 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.083392:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4e00, rc 0) 00000100:00100000:0.0:1769074752.083701:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616600576:12345-192.168.204.60@tcp:101:cat.0 Request processed in 396us (542us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.083708:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13448 00000100:00100000:1.0:1769074752.099938:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.115617:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616600832 00000100:00100000:0.0:1769074752.115631:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616600832 00000100:00100000:0.0:1769074752.115696:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13449 00000100:00100000:0.0:1769074752.115706:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62464:x1855007616600832:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074752.115729:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.115754:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab4b expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.115795:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.115856:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074752.115859:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074752.115863:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab4b expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.115900:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.115919:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 19 pid: 27843 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074752.115933:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625021952:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074752.116015:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a4229c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.125352:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625021952:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074752.125400:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab4b expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.125423:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877000, rc 0) 00000100:00100000:0.0:1769074752.125511:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62464:x1855007616600832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9806us (11186us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.125521:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13449 00000100:00100000:0.0:1769074752.132484:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616601088 00000100:00100000:0.0:1769074752.132499:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616601088 00000100:00100000:0.0:1769074752.132566:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13450 00000100:00100000:0.0:1769074752.132575:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616601088:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074752.132593:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.132609:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd51f88 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab52 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074752.132674:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625022080:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074752.132767:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a4229c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.137055:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616600960:12345-192.168.204.60@tcp:4:cat.0 Request processed in 15697us (16053us total) trans 17179874408 rc 0/0 00000100:00100000:0.0:1769074752.137070:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 656 00002000:00100000:0.0:1769074752.137084:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3416063) req@ffff89b67a0f6d80 x1855007616600960/t17179874408(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/448 e 0 to 0 dl 1769074763 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.137096:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6d80 with x1855007616600960 ext(3145728->3416063) 00010000:00010000:0.0:1769074752.137105:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 24 pid: 27843 timeout: 1839 lvb_type: 0 00010000:00010000:0.0:1769074752.137123:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd51f49 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3416063) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab1a expref: 24 pid: 27843 timeout: 1839 lvb_type: 0 00002000:00010000:0.0:1769074752.137132:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a0f6d80 00000100:00100000:0.0:1769074752.141788:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625022080:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074752.141871:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd51f88 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab52 expref: 23 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.141897:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd51f88 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab52 expref: 23 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.141910:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:0.0:1769074752.141967:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616601088:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9393us (15044us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074752.141978:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13450 00000100:00100000:1.0:1769074752.146334:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.146415:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616601216 00000100:00100000:0.0:1769074752.146426:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616601216 00000100:00100000:0.0:1769074752.146497:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13451 00000100:00100000:0.0:1769074752.146508:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62452:x1855007616601216:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074752.146527:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.146547:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd51f8f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab59 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.146575:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074752.146578:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074752.146583:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd51f8f lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4804608->5242879] (req 4804608->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab59 expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.146596:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd51f8f lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4804608->5242879] (req 4804608->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab59 expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.146609:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:0.0:1769074752.146681:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62452:x1855007616601216:12345-192.168.204.60@tcp:101:cat.0 Request processed in 174us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.146689:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13451 00000100:00100000:1.0:1769074752.151003:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.151089:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616601600 00000100:00100000:0.0:1769074752.151100:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616601600 00000100:00100000:0.0:1769074752.151178:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13179 00000100:00100000:0.0:1769074752.151188:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62466:x1855007616601600:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074752.151212:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.151235:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd51f96 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab60 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.151349:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.151369:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab0c expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.151378:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.151415:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab0c expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.151427:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd51f3b lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab0c expref: 19 pid: 6514 timeout: 1839 lvb_type: 0 00000100:00100000:0.0:1769074752.151440:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625022208:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074752.151536:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b474840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074752.156953:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.157108:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625022208:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074752.157136:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074752.157925:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074752.157933:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074752.157957:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:1.0:1769074752.158118:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd51f96 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab60 expref: 35 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.158140:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab60 expref: 34 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.158148:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074752.158151:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074752.158153:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876600, rc 0) 00000100:00100000:1.0:1769074752.158192:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+34:62466:x1855007616601600:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 7006us (7206us total) trans 8589943770 rc 0/0 00000100:00100000:1.0:1769074752.158200:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13179 00000100:00080000:0.0:1769074752.158581:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074752.161414:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.162190:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616601856 00000100:00100000:0.0:1769074752.162199:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616601856 00002000:00100000:0.0:1769074752.162246:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4804608->4808703) req@ffff89b54cdde300 x1855007616601856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.162256:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cdde300 with x1855007616601856 ext(4804608->4808703) 00002000:00010000:0.0:1769074752.162262:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cdde300 00000100:00100000:0.0:1769074752.162289:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8186 00000100:00100000:0.0:1769074752.162297:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cdde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616601856:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074752.162441:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cdde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616601856:12345-192.168.204.60@tcp:3:cat.0 Request processed in 143us (1044us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074752.162449:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8186 00002000:00100000:0.0:1769074752.162455:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4804608->4808703) req@ffff89b54cdde300 x1855007616601856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/4536 e 0 to 0 dl 1769074763 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.162463:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cdde300 with x1855007616601856 ext(4804608->4808703) 00002000:00010000:0.0:1769074752.162466:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cdde300 00000100:00100000:1.0:1769074752.194866:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.194910:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.292889:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.293404:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616602368 00000100:00100000:0.0:1769074752.293419:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616602368 00000100:00100000:0.0:1769074752.293522:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13180 00000100:00100000:0.0:1769074752.293536:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cdddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62470:x1855007616602368:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074752.293566:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.293593:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd51fab lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab67 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.293701:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293719:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293734:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293753:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293761:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293772:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293915:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876400/0x7fe2323adfd51fc0 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293928:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876400/0x7fe2323adfd51fc0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.293936:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876400/0x7fe2323adfd51fc0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074752.293948:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074752.294048:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294058:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294064:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294072:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294078:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074752.294082:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294090:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294105:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074752.294109:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074752.294116:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294123:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd51fb2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294138:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294144:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294150:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294155:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074752.294157:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294163:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294169:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b65436a700) 00010000:00010000:0.0:1769074752.294172:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074752.294175:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294180:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fb9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074752.294220:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd51fab lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab67 expref: 252 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.294234:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876400/0x7fe2323adfd51fc0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab67 expref: 251 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.294241:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074752.294243:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074752.294245:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876400, rc 0) 00000100:00100000:0.0:1769074752.294307:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cdddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62470:x1855007616602368:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 772us (1424us total) trans 8589943771 rc 0/0 00000100:00100000:0.0:1769074752.294317:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13180 00000100:00100000:0.0:1769074752.309262:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.455955:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.465278:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.466383:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.472105:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.474922:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616603264 00000100:00100000:0.0:1769074752.474948:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616603264 00000100:00100000:0.0:1769074752.475103:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 974 00000100:00100000:0.0:1769074752.475123:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+252:2408:x1855007616603264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074752.475150:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074752.475188:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd51fe3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab6e expref: 252 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.475219:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074752.475226:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074752.475324:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:2408:x1855007616603264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 209us (3231us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.475333:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 974 00000100:00100000:1.0:1769074752.489332:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.490049:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616603392 00000100:00100000:0.0:1769074752.490064:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616603392 00010000:00010000:0.0:1769074752.490120:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd51f8f lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab59 expref: 22 pid: 27843 timeout: 1840 lvb_type: 0 00000100:00100000:0.0:1769074752.490170:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6625 00000100:00100000:0.0:1769074752.490187:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616603392:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074752.490219:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074752.490250:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd51f8f lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab59 expref: 22 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.490268:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd51f8f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab59 expref: 22 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.490304:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074752.490385:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616603392:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 200us (1065us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.490397:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6625 00000100:00100000:1.0:1769074752.492633:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.499895:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.504578:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616603648 00000100:00100000:0.0:1769074752.504603:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616603648 00010000:00010000:1.0:1769074752.509231:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab4b expref: 9 pid: 27843 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.509326:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6626 00000100:00100000:1.0:1769074752.509340:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616603648:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.509361:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.509384:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab4b expref: 9 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.509397:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab4b expref: 9 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.509420:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd51f81 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab4b expref: 9 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.509436:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.509453:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.509461:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.509477:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.509513:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074752.509526:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625022848:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074752.509661:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.529195:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616603904 00000100:00100000:0.0:1769074752.529209:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616603904 00000100:00100000:0.0:1769074752.529283:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13454 00000100:00100000:0.0:1769074752.529292:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62466:x1855007616603904:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074752.529309:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.529326:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ab83 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.529356:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd51f6c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab36 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.529364:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.529377:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.529379:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.529383:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->3858431] (req 3145728->3858431) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 13 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.529419:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.529431:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 13 pid: 36516 timeout: 1840 lvb_type: 1 00000100:00100000:0.0:1769074752.529444:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625022976:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074752.529568:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd51f6c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab36 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.529577:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd51f6c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab36 expref: 18 pid: 27843 timeout: 1840 lvb_type: 0 00000100:00100000:0.0:1769074752.529585:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625023104:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074752.529607:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b474840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.542560:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.542595:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.542697:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616604160 00000100:00100000:1.0:1769074752.542709:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616604160 00010000:00010000:1.0:1769074752.542754:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd51f6c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab36 expref: 19 pid: 27843 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.542851:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616604288 00000100:00100000:1.0:1769074752.542859:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616604288 00010000:00010000:1.0:1769074752.542880:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 14 pid: 36516 timeout: 1840 lvb_type: 1 00000100:00100000:1.0:1769074752.542915:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6627 00000100:00100000:1.0:1769074752.542929:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616604160:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.542952:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.542976:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd51f6c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab36 expref: 19 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.542990:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd51f6c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab36 expref: 19 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.543049:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.543121:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616604160:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 194us (569us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.543131:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6627 00000100:00100000:1.0:1769074752.543151:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6628 00000100:00100000:1.0:1769074752.543158:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616604288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.543165:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.543174:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 14 pid: 36516 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.543188:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd51f7a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab44 expref: 14 pid: 36516 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.543201:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.543203:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.543211:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.543223:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.543260:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074752.543274:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625023360:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074752.543326:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.556691:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.556876:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625023360:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074752.556924:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.557022:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616604288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13860us (14421us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.557036:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6628 00000100:00100000:1.0:1769074752.589457:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616604672 00000100:00100000:1.0:1769074752.589470:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616604672 00010000:00010000:1.0:1769074752.589514:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 10 pid: 36516 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.590483:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6629 00000100:00100000:1.0:1769074752.590492:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b643b0c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616604672:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.590503:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074752.590518:0:8424:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074751 -> 1769074752 00002000:00010000:1.0:1769074752.590521:0:8424:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074751 -> 1769074752 00010000:00010000:1.0:1769074752.590527:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 10 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.590540:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 10 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.590564:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd51ff1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4005888->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab75 expref: 10 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.590582:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.590584:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.590593:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd52006 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab8a expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.590603:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd52006 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab8a expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.590641:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd52006 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab8a expref: 21 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074752.590671:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625023488:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074752.590758:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074752.594866:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625023488:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074752.594892:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.594943:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b643b0c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616604672:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4452us (5946us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.594953:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6629 00000100:00100000:0.0:1769074752.632099:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544685c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616604800:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21785us (22198us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074752.632115:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8191 00002000:00100000:0.0:1769074752.632126:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b544685c00 x1855007616604800/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/440 e 0 to 0 dl 1769074763 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.632137:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544685c00 with x1855007616604800 ext(2097152->3145727) 00002000:00010000:0.0:1769074752.632144:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b544685c00 00000100:00100000:1.0:1769074752.646422:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616605056 00000100:00100000:1.0:1769074752.646429:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616605056 00010000:00010000:1.0:1769074752.646492:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd51d43 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d8 expref: 13 pid: 28954 timeout: 1840 lvb_type: 1 00000100:00100000:1.0:1769074752.646524:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616605184 00000100:00100000:1.0:1769074752.646526:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616605184 00010000:00010000:1.0:1769074752.646537:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeae00/0x7fe2323adfd51d12 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9c3 expref: 9 pid: 27844 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.646555:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6630 00000100:00100000:1.0:1769074752.646562:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616605056:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.646575:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.646594:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd51d43 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d8 expref: 13 pid: 28954 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.646606:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd51d43 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d8 expref: 13 pid: 28954 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.646633:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd51d43 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d8 expref: 13 pid: 28954 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.648797:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074752.653273:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616605056:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6693us (7002us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.653292:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6630 00000100:00100000:0.0:1769074752.655558:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616605312 00000100:00100000:0.0:1769074752.655563:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616605312 00010000:00010000:0.0:1769074752.655584:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd51ddd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa2c expref: 23 pid: 28954 timeout: 1840 lvb_type: 1 00000100:00100000:0.0:1769074752.655616:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6632 00000100:00100000:0.0:1769074752.655623:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616605312:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074752.655633:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074752.655662:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd51ddd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa2c expref: 23 pid: 28954 timeout: 1840 lvb_type: 1 00010000:00010000:0.0:1769074752.655672:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd51ddd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa2c expref: 23 pid: 28954 timeout: 1840 lvb_type: 1 00010000:00010000:0.0:1769074752.655688:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd51ddd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa2c expref: 23 pid: 28954 timeout: 1840 lvb_type: 1 00010000:00010000:0.0:1769074752.655702:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.655703:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.655708:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.655715:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.655739:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074752.655750:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625024128:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074752.655784:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b474840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074752.664420:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.664457:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.666986:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625024128:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074752.667087:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.667189:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616605312:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11563us (12971us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.667201:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6632 00000100:00100000:0.0:1769074752.681214:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.690785:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.690971:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606208 00000100:00100000:1.0:1769074752.690981:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606208 00010000:00010000:1.0:1769074752.691034:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.691081:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6633 00000100:00100000:1.0:1769074752.691092:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616606208:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.691112:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074752.691134:0:8424:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074751 -> 1769074752 00002000:00010000:1.0:1769074752.691138:0:8424:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074751 -> 1769074752 00010000:00010000:1.0:1769074752.691144:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.691159:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.691189:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd51fff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3858431) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab83 expref: 14 pid: 27843 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.691210:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.691213:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.691223:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.691234:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.691288:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074752.691307:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625024384:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074752.691421:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3973c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074752.695565:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625024384:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074752.695594:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.695681:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616606208:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4589us (4907us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.695694:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6633 00000100:00100000:0.0:1769074752.701118:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606336 00000100:00100000:0.0:1769074752.701131:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606336 00000100:00100000:0.0:1769074752.701209:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13461 00000100:00100000:0.0:1769074752.701220:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:62470:x1855007616606336:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074752.701243:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.701262:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879abb4 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.701295:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.701311:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 9 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.701319:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 20 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.701326:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.701328:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.701332:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->1605631] (req 1048576->1605631) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.701368:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 20 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.701381:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 20 pid: 8396 timeout: 1840 lvb_type: 1 00000100:00100000:0.0:1769074752.701394:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625024512:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074752.701514:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 9 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.701523:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 9 pid: 27843 timeout: 1840 lvb_type: 1 00000100:00100000:0.0:1769074752.701532:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625024640:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074752.701565:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.701574:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 12 pid: 27844 timeout: 1840 lvb_type: 0 00000100:00100000:0.0:1769074752.701582:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625024768:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074752.701603:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b474840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.704414:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074752.704538:0:8402:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606464 00000100:00100000:1.0:1769074752.704550:0:8402:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606464 00002000:00100000:1.0:1769074752.704611:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2bc:0x0] (593920->1048575) req@ffff89b54ce9ca80 x1855007616606464/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.704630:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ca80 with x1855007616606464 ext(593920->1048575) 00010000:00010000:1.0:1769074752.704640:0:8402:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 22 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.704671:0:8402:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 22 pid: 36516 timeout: 1840 lvb_type: 0 00002000:00010000:1.0:1769074752.704677:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9ca80 00000100:00100000:1.0:1769074752.704713:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 660 00000100:00100000:1.0:1769074752.704722:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616606464:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074752.710128:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606592 00000100:00100000:1.0:1769074752.710137:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606592 00010000:00010000:1.0:1769074752.710177:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 21 pid: 8396 timeout: 1840 lvb_type: 1 00000100:00100000:1.0:1769074752.710220:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6634 00000100:00100000:1.0:1769074752.710229:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616606592:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.710244:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.710268:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 21 pid: 8396 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.710281:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 21 pid: 8396 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.710309:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.710378:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616606592:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 149us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.710386:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6634 00010000:00010000:0.0:1769074752.710997:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd51de4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa33 expref: 20 pid: 8396 timeout: 1840 lvb_type: 1 00000100:00100000:0.0:1769074752.711014:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625024512:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074752.719577:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625024640:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074752.719592:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625024768:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074752.719632:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->1605631] (req 1048576->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.719660:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546f90a00, rc 0) 00000100:00100000:0.0:1769074752.719709:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:62470:x1855007616606336:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 18491us (18785us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.719718:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13461 00000100:00100000:1.0:1769074752.719945:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606848 00000100:00100000:1.0:1769074752.719953:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606848 00010000:00010000:1.0:1769074752.719992:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 13 pid: 27844 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.720022:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606720 00000100:00100000:1.0:1769074752.720026:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606720 00010000:00010000:1.0:1769074752.720054:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 10 pid: 27843 timeout: 1840 lvb_type: 1 00000100:00100000:1.0:1769074752.720076:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6635 00000100:00100000:1.0:1769074752.720084:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616606848:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.720099:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.720121:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 13 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.720131:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 13 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.720155:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeb000/0x7fe2323adfd51d35 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a9d1 expref: 13 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.720171:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.720194:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616606848:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 111us (839us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.720203:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6635 00000100:00100000:1.0:1769074752.720220:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6636 00000100:00100000:1.0:1769074752.720225:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b643b0fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616606720:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.720234:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074752.720242:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 10 pid: 27843 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.720251:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 10 pid: 27843 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.720263:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd51ee7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aacd expref: 10 pid: 27843 timeout: 1840 lvb_type: 1 00010000:00010000:1.0:1769074752.720271:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.720273:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.720280:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.720291:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.720325:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074752.720341:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625024896:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074752.720403:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3973c0 going to sleep for 11 seconds 00000100:00080000:1.0:1769074752.720423:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074752.725370:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616606976 00000100:00100000:1.0:1769074752.725382:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616606976 00002000:00100000:1.0:1769074752.725435:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54c845500 x1855007616606976/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.725467:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845500 with x1855007616606976 ext(2097152->4194303) 00002000:00010000:1.0:1769074752.725475:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c845500 00000100:00100000:1.0:1769074752.725508:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616607104 00000100:00100000:1.0:1769074752.725512:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616607104 00002000:00100000:1.0:1769074752.725529:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4808703) req@ffff89b5436d7100 x1855007616607104/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.725537:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7100 with x1855007616607104 ext(3145728->4808703) 00002000:00010000:1.0:1769074752.725540:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d7100 00000100:00100000:1.0:1769074752.725567:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8192 00000100:00100000:1.0:1769074752.725576:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616606976:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074752.758423:0:8402:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616606464:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 53690us (54011us total) trans 17179874410 rc 0/0 00000100:00100000:1.0:1769074752.758716:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 660 00002000:00100000:1.0:1769074752.758735:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2bc:0x0] (593920->1048575) req@ffff89b54ce9ca80 x1855007616606464/t17179874410(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/448 e 0 to 0 dl 1769074763 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.758745:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ca80 with x1855007616606464 ext(593920->1048575) 00010000:00010000:1.0:1769074752.758755:0:8402:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 26 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.758771:0:8402:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 26 pid: 36516 timeout: 1840 lvb_type: 0 00002000:00010000:1.0:1769074752.758780:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9ca80 00000100:00100000:1.0:1769074752.772001:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616606976:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 46404us (46750us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074752.772019:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8192 00002000:00100000:1.0:1769074752.772033:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54c845500 x1855007616606976/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/440 e 0 to 0 dl 1769074763 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.772044:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845500 with x1855007616606976 ext(2097152->4194303) 00002000:00010000:1.0:1769074752.772051:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c845500 00000100:00100000:1.0:1769074752.796706:0:24026:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616607232 00000100:00100000:1.0:1769074752.796720:0:24026:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616607232 00002000:00100000:1.0:1769074752.796790:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2ba:0x0] (1601536->1605631) req@ffff89b54920a680 x1855007616607232/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.796850:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920a680 with x1855007616607232 ext(1601536->1605631) 00002000:00010000:1.0:1769074752.796859:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54920a680 00000100:00100000:1.0:1769074752.796891:0:24026:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8194 00000100:00100000:1.0:1769074752.796900:0:24026:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616607232:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074752.797151:0:24026:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616607232:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 251us (762us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074752.797161:0:24026:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8194 00002000:00100000:1.0:1769074752.797169:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2ba:0x0] (1601536->1605631) req@ffff89b54920a680 x1855007616607232/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/4536 e 0 to 0 dl 1769074763 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.797177:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920a680 with x1855007616607232 ext(1601536->1605631) 00002000:00010000:1.0:1769074752.797182:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54920a680 00000100:00100000:1.0:1769074752.808346:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616607360 00000100:00100000:1.0:1769074752.808361:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616607360 00010000:00010000:1.0:1769074752.808413:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 24 pid: 36516 timeout: 1840 lvb_type: 0 00000100:00100000:1.0:1769074752.808470:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6637 00000100:00100000:1.0:1769074752.808478:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616607360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074752.808494:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074752.808510:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074749 -> 1769074752 00002000:00010000:1.0:1769074752.808513:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074749 -> 1769074752 00010000:00010000:1.0:1769074752.808520:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 24 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.808532:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 24 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.808554:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd5200d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 593920->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab91 expref: 24 pid: 36516 timeout: 1840 lvb_type: 0 00010000:00010000:1.0:1769074752.808570:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.808572:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074752.808579:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 9 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.808589:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 9 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074752.808631:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 9 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074752.808653:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542faa680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625025024:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074752.808773:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074752.819188:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542faa680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625025024:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074752.819225:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074752.819290:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616607360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10810us (11515us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074752.819302:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6637 00000100:00100000:0.0:1769074752.837217:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616607616 00000100:00100000:0.0:1769074752.837223:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616607616 00000100:00100000:0.0:1769074752.837263:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13462 00000100:00100000:0.0:1769074752.837273:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616607616:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074752.837296:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074752.837320:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879abbb expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.837341:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.837343:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.837353:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 24 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.837370:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 24 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074752.837383:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546f91a00, rc 0) 00000100:00100000:0.0:1769074752.837442:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616607616:12345-192.168.204.60@tcp:101:cat.0 Request processed in 170us (554us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.837449:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13462 00000100:00100000:1.0:1769074752.845250:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616607488 00000100:00100000:1.0:1769074752.845269:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616607488 00002000:00100000:1.0:1769074752.845329:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54cfed500 x1855007616607488/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074752.845340:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed500 with x1855007616607488 ext(0->1048575) 00002000:00010000:1.0:1769074752.845346:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfed500 00000100:00100000:1.0:1769074752.845389:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8195 00000100:00100000:1.0:1769074752.845398:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616607488:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074752.878542:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616607488:12345-192.168.204.60@tcp:3:cat.0 Request processed in 33128us (41548us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074752.878559:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8195 00002000:00100000:0.0:1769074752.878576:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54cfed500 x1855007616607488/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:288/0 lens 488/440 e 0 to 0 dl 1769074763 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.878588:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed500 with x1855007616607488 ext(0->1048575) 00002000:00010000:0.0:1769074752.878595:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfed500 00000100:00100000:0.0:1769074752.895710:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074752.904222:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616608128 00000100:00100000:0.0:1769074752.904233:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616608128 00002000:00100000:0.0:1769074752.904287:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1605631) req@ffff89b54cfec380 x1855007616608128/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.904298:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfec380 with x1855007616608128 ext(1048576->1605631) 00010000:00010000:0.0:1769074752.904308:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 19 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.904323:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 19 pid: 27844 timeout: 1840 lvb_type: 0 00002000:00010000:0.0:1769074752.904331:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cfec380 00000100:00100000:0.0:1769074752.904369:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 661 00000100:00100000:0.0:1769074752.904378:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616608128:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074752.933546:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616608128:12345-192.168.204.60@tcp:4:cat.0 Request processed in 29150us (29486us total) trans 21474841093 rc 0/0 00000100:00100000:0.0:1769074752.933564:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 661 00002000:00100000:0.0:1769074752.933580:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1605631) req@ffff89b54cfec380 x1855007616608128/t21474841093(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:288/0 lens 488/448 e 0 to 0 dl 1769074763 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.933592:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfec380 with x1855007616608128 ext(1048576->1605631) 00010000:00010000:0.0:1769074752.933604:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 21 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.933621:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 21 pid: 27844 timeout: 1840 lvb_type: 0 00002000:00010000:0.0:1769074752.933635:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cfec380 00000100:00100000:1.0:1769074752.945309:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616608256 00000100:00100000:1.0:1769074752.945325:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616608256 00010000:00010000:0.0:1769074752.945483:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 21 pid: 27844 timeout: 1840 lvb_type: 0 00000100:00100000:0.0:1769074752.945537:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6638 00000100:00100000:0.0:1769074752.945547:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616608256:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074752.945562:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1769074752.945580:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074749 -> 1769074752 00002000:00010000:0.0:1769074752.945583:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074749 -> 1769074752 00010000:00010000:0.0:1769074752.945589:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 21 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.945604:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 21 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.945633:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52030 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1605631) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abb4 expref: 21 pid: 27844 timeout: 1840 lvb_type: 0 00010000:00010000:0.0:1769074752.945659:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.945662:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074752.945670:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.945680:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074752.945724:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074752.945737:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625025280:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074752.945868:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b474840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074752.951633:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625025280:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074752.951677:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074752.951740:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616608256:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6193us (9036us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074752.951752:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6638 00000100:00100000:0.0:1769074752.960367:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616608384 00000100:00100000:0.0:1769074752.960379:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616608384 00002000:00100000:0.0:1769074752.960435:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cfeca80 x1855007616608384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:288/0 lens 488/0 e 0 to 0 dl 1769074763 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.960444:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfeca80 with x1855007616608384 ext(1048576->2097151) 00002000:00010000:0.0:1769074752.960450:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfeca80 00000100:00100000:0.0:1769074752.960498:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8196 00000100:00100000:0.0:1769074752.960508:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616608384:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074752.977285:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616608384:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16760us (17259us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074752.977304:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8196 00002000:00100000:0.0:1769074752.977323:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cfeca80 x1855007616608384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:288/0 lens 488/440 e 0 to 0 dl 1769074763 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074752.977342:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfeca80 with x1855007616608384 ext(1048576->2097151) 00002000:00010000:0.0:1769074752.977350:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfeca80 00000100:00100000:0.0:1769074753.018066:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616608512 00000100:00100000:0.0:1769074753.018083:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616608512 00002000:00100000:0.0:1769074753.018148:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b545108000 x1855007616608512/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.018159:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108000 with x1855007616608512 ext(2097152->3145727) 00002000:00010000:0.0:1769074753.018166:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545108000 00000100:00100000:0.0:1769074753.018191:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8197 00000100:00100000:0.0:1769074753.018200:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616608512:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074753.046525:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616608512:12345-192.168.204.60@tcp:3:cat.0 Request processed in 28307us (34981us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074753.046545:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8197 00002000:00100000:0.0:1769074753.046563:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b545108000 x1855007616608512/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/440 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.046576:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108000 with x1855007616608512 ext(2097152->3145727) 00002000:00010000:0.0:1769074753.046583:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545108000 00000100:00100000:0.0:1769074753.062150:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616608896 00000100:00100000:0.0:1769074753.062162:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616608896 00000100:00100000:0.0:1769074753.062256:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 975 00000100:00100000:0.0:1769074753.062271:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2408:x1855007616608896:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.062293:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.062325:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5205a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc9 expref: 227 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.062348:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.062354:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.062449:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2408:x1855007616608896:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 183us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.062456:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 975 00000100:00100000:0.0:1769074753.067706:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616609536 00000100:00100000:0.0:1769074753.067720:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616609536 00000100:00100000:0.0:1769074753.067871:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13183 00000100:00100000:0.0:1769074753.067886:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62489:x1855007616609536:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074753.067912:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074753.067940:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52068 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879abd7 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.068535:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068556:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068574:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068594:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068604:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068612:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068766:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5207d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068780:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5207d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.068789:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5207d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074753.068802:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:1.0:1769074753.068977:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069008:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069017:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069024:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069030:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074753.069035:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069043:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069057:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074753.069061:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074753.069068:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069075:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5206f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069092:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069099:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069105:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069110:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba44000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074753.069113:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069119:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069127:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294700) 00010000:00010000:1.0:1769074753.069131:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074753.069134:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069144:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52076 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074753.069208:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90a00/0x7fe2323adfd52068 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd7 expref: 230 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.069228:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5207d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd7 expref: 229 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.069237:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074753.069240:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074753.069243:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00000100:00100000:1.0:1769074753.069611:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62489:x1855007616609536:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1726us (2183us total) trans 8589943777 rc 0/0 00000100:00100000:1.0:1769074753.069623:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13183 00000100:00080000:1.0:1769074753.077417:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074753.078855:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616609792 00000100:00100000:0.0:1769074753.078866:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616609792 00000100:00100000:0.0:1769074753.078951:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 976 00000100:00100000:0.0:1769074753.078966:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:2406:x1855007616609792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.079008:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.079042:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5207d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd7 expref: 229 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.079063:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.079070:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.079138:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:2406:x1855007616609792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 176us (733us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.079151:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 976 00000100:00100000:0.0:1769074753.085133:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616609664 00000100:00100000:0.0:1769074753.085147:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616609664 00000100:00100000:0.0:1769074753.085212:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13465 00000100:00100000:0.0:1769074753.085223:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616609664:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074753.085246:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074753.085266:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879abde expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.085295:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.085298:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.085308:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abde expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.085326:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abde expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.085340:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546f90e00, rc 0) 00000100:00100000:0.0:1769074753.085411:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616609664:12345-192.168.204.60@tcp:101:cat.0 Request processed in 189us (714us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.085419:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13465 00000100:00100000:1.0:1769074753.123730:0:24026:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610048 00000100:00100000:1.0:1769074753.123741:0:24026:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610048 00002000:00100000:1.0:1769074753.123789:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54d100000 x1855007616610048/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.123801:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100000 with x1855007616610048 ext(1048576->2097151) 00002000:00010000:1.0:1769074753.123841:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d100000 00000100:00100000:1.0:1769074753.123862:0:24026:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616609920 00000100:00100000:1.0:1769074753.123865:0:24026:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616609920 00002000:00100000:1.0:1769074753.123901:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5461e7800 x1855007616609920/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.123910:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7800 with x1855007616609920 ext(0->1048575) 00002000:00010000:1.0:1769074753.123915:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e7800 00000100:00100000:1.0:1769074753.123935:0:24026:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8198 00000100:00100000:1.0:1769074753.123943:0:24026:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616610048:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074753.124551:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8199 00000100:00100000:1.0:1769074753.124561:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616609920:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074753.152789:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610176 00000100:00100000:0.0:1769074753.152836:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610176 00002000:00100000:0.0:1769074753.152930:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5497aaa00 x1855007616610176/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.152942:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aaa00 with x1855007616610176 ext(2097152->4194303) 00002000:00010000:0.0:1769074753.152949:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497aaa00 00000100:00100000:0.0:1769074753.152986:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610304 00000100:00100000:0.0:1769074753.152991:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610304 00002000:00100000:0.0:1769074753.153008:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5497a9180 x1855007616610304/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.153016:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9180 with x1855007616610304 ext(3145728->4194303) 00002000:00010000:0.0:1769074753.153020:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a9180 00000100:00100000:0.0:1769074753.153039:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8200 00000100:00100000:0.0:1769074753.153049:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616610176:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074753.166239:0:24026:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616610048:12345-192.168.204.60@tcp:3:cat.0 Request processed in 42283us (42650us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074753.166254:0:24026:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8198 00002000:00100000:1.0:1769074753.166269:0:24026:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54d100000 x1855007616610048/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/440 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.166280:0:24026:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100000 with x1855007616610048 ext(1048576->2097151) 00002000:00010000:1.0:1769074753.166285:0:24026:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d100000 00000100:00100000:1.0:1769074753.179200:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610432 00000100:00100000:1.0:1769074753.179211:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610432 00000100:00100000:1.0:1769074753.179271:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13466 00000100:00100000:1.0:1769074753.179280:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62484:x1855007616610432:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074753.179297:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074753.179316:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879abe5 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.179344:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.179353:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 8 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074753.179361:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 20 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074753.179367:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074753.179369:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074753.179373:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1425408->2097151] (req 1425408->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.179406:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 20 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074753.179419:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 20 pid: 36516 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.179432:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542faa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625025408:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.179540:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 8 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074753.179550:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 8 pid: 27843 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.179560:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625025536:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.179589:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.179603:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 17 pid: 36516 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.179611:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625025664:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074753.179632:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074753.192001:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616609920:12345-192.168.204.60@tcp:3:cat.0 Request processed in 67414us (68341us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074753.192017:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8199 00002000:00100000:1.0:1769074753.192032:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5461e7800 x1855007616609920/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/440 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.192044:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7800 with x1855007616609920 ext(0->1048575) 00002000:00010000:1.0:1769074753.192050:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e7800 00000100:00100000:0.0:1769074753.205932:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.207608:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616610176:12345-192.168.204.60@tcp:3:cat.0 Request processed in 54552us (69877us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074753.207622:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8200 00002000:00100000:1.0:1769074753.207644:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5497aaa00 x1855007616610176/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/440 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.207653:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aaa00 with x1855007616610176 ext(2097152->4194303) 00002000:00010000:1.0:1769074753.207660:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497aaa00 00000100:00100000:0.0:1769074753.210095:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.210162:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.210189:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610688 00000100:00100000:0.0:1769074753.210198:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610688 00010000:00010000:0.0:1769074753.210235:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 21 pid: 36516 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.210264:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610816 00000100:00100000:0.0:1769074753.210266:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610816 00010000:00010000:0.0:1769074753.210280:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 9 pid: 27843 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.210296:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6639 00000100:00100000:0.0:1769074753.210304:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616610688:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.210319:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.210342:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 21 pid: 36516 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.210355:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 21 pid: 36516 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.210383:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.210434:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616610688:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 130us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.210441:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6639 00000100:00100000:0.0:1769074753.210472:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6640 00000100:00100000:0.0:1769074753.210479:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616610816:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.210487:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.210495:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 9 pid: 27843 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.210504:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 9 pid: 27843 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.210516:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.210532:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616610816:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 54us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.210538:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6640 00010000:00010000:1.0:1769074753.210948:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd51ebd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aab8 expref: 20 pid: 36516 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.210963:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542faa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625025408:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.210976:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd51eee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aad4 expref: 8 pid: 27843 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.210984:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625025536:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074753.210993:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625025664:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.211023:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1425408->2097151] (req 1425408->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.211035:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44000, rc 0) 00000100:00100000:1.0:1769074753.211076:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62484:x1855007616610432:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 31797us (32058us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.211084:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13466 00000100:00100000:1.0:1769074753.222991:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616610944 00000100:00100000:1.0:1769074753.223002:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616610944 00010000:00010000:1.0:1769074753.223040:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 15 pid: 36516 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.223080:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6641 00000100:00100000:1.0:1769074753.223090:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616610944:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074753.223104:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074753.223124:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 15 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.223138:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 15 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.223168:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd51e54 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa95 expref: 15 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.223187:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074753.223189:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074753.223198:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 14 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.223209:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 14 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.223250:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 14 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074753.223264:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625026176:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074753.223338:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397240 going to sleep for 11 seconds 00000100:00100000:0.0:1769074753.229237:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.232283:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625026176:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074753.232313:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074753.232374:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616610944:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9285us (9611us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.232387:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6641 00000100:00100000:0.0:1769074753.234207:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.239689:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.240915:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.245121:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616611200 00000100:00100000:1.0:1769074753.245137:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616611200 00010000:00010000:1.0:1769074753.245188:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 10 pid: 36516 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.245237:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616611328 00000100:00100000:1.0:1769074753.245241:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616611328 00010000:00010000:1.0:1769074753.245253:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d400/0x7fe2323adfd51ec4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aabf expref: 25 pid: 27843 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.245288:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6643 00000100:00100000:1.0:1769074753.245296:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616611200:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074753.245310:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074753.245330:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 10 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.245342:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 10 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.245365:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c800/0x7fe2323adfd51e5b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aa9c expref: 10 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.245383:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074753.247363:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616611200:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 143us (11250us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.247375:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6643 00000100:00100000:0.0:1769074753.259370:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.259488:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.260714:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616611712 00000100:00100000:1.0:1769074753.260723:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616611712 00000100:00100000:1.0:1769074753.260770:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13469 00000100:00100000:1.0:1769074753.260778:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62491:x1855007616611712:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074753.260789:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074753.260840:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879abfa expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.260868:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 13 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.260877:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074753.260880:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074753.260884:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abfa expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.260925:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 13 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.260939:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 13 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.260953:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625026560:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074753.261029:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074753.274264:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625026560:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.274302:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abfa expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.274322:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45800, rc 0) 00000100:00100000:1.0:1769074753.274382:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62491:x1855007616611712:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13604us (15020us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.274391:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13469 00000100:00100000:1.0:1769074753.285157:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616611968 00000100:00100000:1.0:1769074753.285168:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616611968 00002000:00100000:1.0:1769074753.285219:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5497a9c00 x1855007616611968/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.285230:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9c00 with x1855007616611968 ext(4194304->4808703) 00002000:00010000:1.0:1769074753.285236:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a9c00 00000100:00100000:1.0:1769074753.285266:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616612096 00000100:00100000:1.0:1769074753.285270:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616612096 00002000:00100000:1.0:1769074753.285284:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x2be:0x0] (2035712->2039807) req@ffff89b5497aa300 x1855007616612096/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.285291:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aa300 with x1855007616612096 ext(2035712->2039807) 00002000:00010000:1.0:1769074753.285295:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aa300 00000100:00100000:1.0:1769074753.285311:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616612352 00000100:00100000:1.0:1769074753.285314:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616612352 00002000:00100000:1.0:1769074753.285330:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x2bb:0x0] (1425408->2097151) req@ffff89b5497a8380 x1855007616612352/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.285337:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8380 with x1855007616612352 ext(1425408->2097151) 00010000:00010000:1.0:1769074753.285345:0:28444:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 14 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.285359:0:28444:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 14 pid: 28954 timeout: 1841 lvb_type: 0 00002000:00010000:1.0:1769074753.285365:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5497a8380 00000100:00100000:1.0:1769074753.285385:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 662 00000100:00100000:1.0:1769074753.285394:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616612352:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074753.285576:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8204 00000100:00100000:1.0:1769074753.285584:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616611968:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074753.286010:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8205 00000100:00100000:1.0:1769074753.286017:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616612096:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074753.286096:0:8402:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616612096:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 79us (1098us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074753.286103:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8205 00002000:00100000:1.0:1769074753.286111:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2be:0x0] (2035712->2039807) req@ffff89b5497aa300 x1855007616612096/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/4536 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.286119:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aa300 with x1855007616612096 ext(2035712->2039807) 00002000:00010000:1.0:1769074753.286123:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aa300 00000100:00100000:1.0:1769074753.286148:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616612224 00000100:00100000:1.0:1769074753.286153:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616612224 00000100:00100000:1.0:1769074753.286181:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13470 00000100:00100000:1.0:1769074753.286187:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62484:x1855007616612224:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074753.286196:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074753.286215:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ac01 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.286242:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd520a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->2039807) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abec expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.286253:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074753.286255:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074753.286259:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->1376255] (req 1048576->1376255) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.286288:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd520a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->2039807) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abec expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.286298:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd520a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->2039807) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abec expref: 23 pid: 27844 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.286310:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625026688:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074753.286362:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397f00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074753.295384:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625026688:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.295416:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->1376255] (req 1048576->1376255) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.295434:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45200, rc 0) 00000100:00100000:1.0:1769074753.295501:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62484:x1855007616612224:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 9313us (10468us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.295510:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13470 00000100:00100000:1.0:1769074753.305757:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616612352:12345-192.168.204.60@tcp:4:cat.0 Request processed in 20347us (20678us total) trans 21474841094 rc 0/0 00000100:00100000:1.0:1769074753.305773:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 662 00002000:00100000:1.0:1769074753.305787:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x2bb:0x0] (1425408->2097151) req@ffff89b5497a8380 x1855007616612352/t21474841094(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/448 e 0 to 0 dl 1769074764 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.305799:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8380 with x1855007616612352 ext(1425408->2097151) 00010000:00010000:1.0:1769074753.305856:0:28444:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 16 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:1.0:1769074753.305875:0:28444:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1425408->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abe5 expref: 16 pid: 28954 timeout: 1841 lvb_type: 0 00002000:00010000:1.0:1769074753.305882:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5497a8380 00000100:00100000:0.0:1769074753.318785:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616612736 00000100:00100000:0.0:1769074753.318797:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616612736 00000100:00100000:0.0:1769074753.318875:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4584 00000100:00100000:0.0:1769074753.318884:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:62489:x1855007616612736:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074753.319143:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62489:x1855007616612736:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 260us (714us total) trans 8589943778 rc 0/0 00000100:00100000:0.0:1769074753.319151:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4584 00000100:00100000:1.0:1769074753.320468:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616611968:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34881us (35530us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074753.320478:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8204 00002000:00100000:1.0:1769074753.320487:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5497a9c00 x1855007616611968/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/440 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.320495:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9c00 with x1855007616611968 ext(4194304->4808703) 00002000:00010000:1.0:1769074753.320499:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a9c00 00000100:00100000:0.0:1769074753.322924:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.342394:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625026816:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074753.342428:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.342494:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616612864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 19384us (19577us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.342506:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6645 00000100:00100000:1.0:1769074753.343290:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616613248 00000100:00100000:1.0:1769074753.343301:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616613248 00002000:00100000:1.0:1769074753.343348:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67a271880 x1855007616613248/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.343358:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271880 with x1855007616613248 ext(1048576->2097151) 00002000:00010000:1.0:1769074753.343364:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a271880 00000100:00100000:1.0:1769074753.343394:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8206 00000100:00100000:1.0:1769074753.343404:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616613248:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074753.363972:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.369147:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.370047:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616613248:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 26642us (27786us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074753.370057:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8206 00002000:00100000:1.0:1769074753.370070:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67a271880 x1855007616613248/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:289/0 lens 488/440 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074753.370079:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271880 with x1855007616613248 ext(1048576->2097151) 00002000:00010000:1.0:1769074753.370084:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a271880 00000100:00100000:0.0:1769074753.380729:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625027072:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074753.380775:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.380879:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616613376:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16689us (16914us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.380894:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6646 00000100:00100000:0.0:1769074753.381882:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616613760 00000100:00100000:0.0:1769074753.381891:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616613760 00002000:00100000:0.0:1769074753.381938:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1372160->1376255) req@ffff89b67fa23100 x1855007616613760/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.381949:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa23100 with x1855007616613760 ext(1372160->1376255) 00002000:00010000:0.0:1769074753.381955:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa23100 00000100:00100000:0.0:1769074753.381989:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8207 00000100:00100000:0.0:1769074753.381999:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616613760:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074753.382353:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616613760:12345-192.168.204.60@tcp:3:cat.0 Request processed in 354us (1079us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074753.382361:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8207 00002000:00100000:0.0:1769074753.382369:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1372160->1376255) req@ffff89b67fa23100 x1855007616613760/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/4536 e 0 to 0 dl 1769074764 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.382376:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa23100 with x1855007616613760 ext(1372160->1376255) 00002000:00010000:0.0:1769074753.382380:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa23100 00000100:00100000:1.0:1769074753.425124:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.425236:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616614016 00000100:00100000:0.0:1769074753.425246:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616614016 00002000:00100000:0.0:1769074753.425301:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1376255) req@ffff89b542fa9500 x1855007616614016/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/0 e 0 to 0 dl 1769074764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.425313:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9500 with x1855007616614016 ext(1048576->1376255) 00010000:00010000:0.0:1769074753.425323:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 10 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.425338:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 10 pid: 28954 timeout: 1841 lvb_type: 0 00002000:00010000:0.0:1769074753.425345:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b542fa9500 00000100:00100000:0.0:1769074753.425385:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 664 00000100:00100000:0.0:1769074753.425395:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616614016:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074753.433866:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616614016:12345-192.168.204.60@tcp:4:cat.0 Request processed in 8458us (8740us total) trans 17179874412 rc 0/0 00000100:00100000:0.0:1769074753.433881:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 664 00002000:00100000:0.0:1769074753.433895:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1376255) req@ffff89b542fa9500 x1855007616614016/t17179874412(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:289/0 lens 488/448 e 0 to 0 dl 1769074764 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074753.433905:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9500 with x1855007616614016 ext(1048576->1376255) 00010000:00010000:0.0:1769074753.433916:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.433935:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 1841 lvb_type: 0 00002000:00010000:0.0:1769074753.433942:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b542fa9500 00000100:00100000:0.0:1769074753.439182:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625027584 00000100:00100000:0.0:1769074753.439190:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625027584 00000100:00100000:0.0:1769074753.439237:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2640 00000100:00100000:0.0:1769074753.439247:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625027584:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074753.439279:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6438912 pending 0 free 1446150144 avail 1299910656 00000100:00100000:0.0:1769074753.439324:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625027584:12345-0@lo:41:osp-pre-1-0.0 Request processed in 78us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.439331:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2640 00000100:00100000:1.0:1769074753.439699:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.441330:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625027712:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441378:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.441423:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625028224:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441460:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.441480:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625028736:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441499:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.441518:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1769074753.441524:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625028096:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441552:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.441571:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625028608:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441591:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.441620:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625028224 00000100:00100000:1.0:1769074753.441627:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625028224 00000100:00100000:1.0:1769074753.441678:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625028608 00000100:00100000:1.0:1769074753.441681:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625028608 00000100:00100000:1.0:1769074753.441697:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2273 00000100:00100000:1.0:1769074753.441703:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625028224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441751:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625028224:12345-0@lo:400:kworker.0 Request processed in 48us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.441757:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2273 00000100:00100000:1.0:1769074753.441773:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2274 00000100:00100000:1.0:1769074753.441779:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625028608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441838:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625028608:12345-0@lo:400:kworker.0 Request processed in 59us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.441843:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2274 00000100:00080000:1.0:1769074753.441863:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074753.441868:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074753.441888:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625028608:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441898:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625028224:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.441928:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625028736 00000100:00100000:1.0:1769074753.441931:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625028736 00000100:00100000:1.0:1769074753.441958:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625028096 00000100:00100000:1.0:1769074753.441961:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625028096 00000100:00100000:1.0:1769074753.441998:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1423 00000100:00100000:1.0:1769074753.442004:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625028736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.442040:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625028736:12345-0@lo:400:kworker.0 Request processed in 36us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.442046:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1423 00000100:00100000:1.0:1769074753.442056:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1424 00000100:00100000:1.0:1769074753.442063:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625028096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.442090:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625028096:12345-0@lo:400:kworker.0 Request processed in 28us (538us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.442096:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1424 00000100:00080000:1.0:1769074753.442115:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074753.442121:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074753.442137:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625028096:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.442146:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841076 gen 2 00000004:00080000:1.0:1769074753.442151:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ef480, transno 21474841053 00000004:00080000:1.0:1769074753.442161:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460eed80, transno 21474841054 00000004:00080000:1.0:1769074753.442164:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ef100, transno 21474841055 00000004:00080000:1.0:1769074753.442166:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ee680, transno 21474841056 00000004:00080000:1.0:1769074753.442169:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ee300, transno 21474841057 00000004:00080000:1.0:1769074753.442171:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ed180, transno 21474841058 00000004:00080000:1.0:1769074753.442174:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460efb80, transno 21474841059 00000004:00080000:1.0:1769074753.442176:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ec380, transno 21474841060 00000004:00080000:1.0:1769074753.442179:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fb97b80, transno 21474841061 00000004:00080000:1.0:1769074753.442181:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fa20000, transno 21474841062 00000004:00080000:1.0:1769074753.442183:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fb95c00, transno 21474841063 00000004:00080000:1.0:1769074753.442186:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fa21c00, transno 21474841064 00000004:00080000:1.0:1769074753.442189:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fa21180, transno 21474841065 00000004:00080000:1.0:1769074753.442192:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d1dc000, transno 21474841066 00000004:00080000:1.0:1769074753.442195:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fb94e00, transno 21474841067 00000004:00080000:1.0:1769074753.442198:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d1dce00, transno 21474841068 00000004:00080000:1.0:1769074753.442201:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54475a680, transno 21474841069 00000004:00080000:1.0:1769074753.442204:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54475bb80, transno 21474841070 00000004:00080000:1.0:1769074753.442207:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c69880, transno 21474841071 00000004:00080000:1.0:1769074753.442209:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54475aa00, transno 21474841072 00000004:00080000:1.0:1769074753.442212:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54475b800, transno 21474841073 00000004:00080000:1.0:1769074753.442214:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c6b800, transno 21474841074 00000004:00080000:1.0:1769074753.442217:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b544759c00, transno 21474841075 00000004:00080000:1.0:1769074753.442220:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c69f80, transno 21474841076 00000100:00100000:1.0:1769074753.442224:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625028736:0@lo:400:kworker.0 00000004:00080000:0.0:1769074753.443280:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 24 done 00000100:00100000:0.0:1769074753.443299:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616614144 00000100:00100000:0.0:1769074753.443304:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616614144 00010000:00010000:0.0:1769074753.443342:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.443381:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6647 00000100:00100000:0.0:1769074753.443388:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616614144:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.443401:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.443427:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.443456:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.443482:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd520b5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1376255) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac01 expref: 12 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.443498:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.443500:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.443507:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.443517:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.443556:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074753.443570:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625029120:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074753.443685:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074753.449499:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625027712:0@lo:400:kworker.0 00000100:00100000:1.0:1769074753.450948:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.451073:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625029120:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074753.451095:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.451163:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616614144:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7774us (11467us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.451174:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6647 00000100:00100000:1.0:1769074753.513383:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.520205:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616614656 00000100:00100000:0.0:1769074753.520215:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616614656 00000100:00100000:0.0:1769074753.520278:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4585 00000100:00100000:0.0:1769074753.520286:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b68164e450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:60112:x1855007616614656:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:0.0:1769074753.520583:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b68164e450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:60112:x1855007616614656:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 295us (1401us total) trans 8589943779 rc 0/0 00000100:00100000:0.0:1769074753.520590:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4585 00000100:00100000:0.0:1769074753.523062:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:0.0:1769074753.524223:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab60 expref: 13 pid: 6514 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.524318:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6648 00000100:00100000:0.0:1769074753.524328:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616614784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.524346:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.524353:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab60 expref: 13 pid: 6514 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.524366:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab60 expref: 13 pid: 6514 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.524389:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd51f9d lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab60 expref: 13 pid: 6514 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.524407:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.524412:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd520ed lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.524424:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd520ed lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.524433:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd520ed lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.524488:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.524560:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616614784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 232us (1507us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.524568:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6648 00000100:00100000:0.0:1769074753.547439:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616615936 00000100:00100000:0.0:1769074753.547458:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616615936 00010000:00010000:0.0:1769074753.547497:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd520ed lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac24 expref: 10 pid: 6512 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.547539:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6649 00000100:00100000:0.0:1769074753.547548:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616615936:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.547561:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.547567:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd520ed lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac24 expref: 10 pid: 6512 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.547578:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd520ed lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac24 expref: 10 pid: 6512 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.547605:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.547610:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52125 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.547621:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52125 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.547638:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52125 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.547657:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.547713:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616615936:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 167us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.547721:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6649 00000100:00100000:0.0:1769074753.558331:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616616192 00000100:00100000:0.0:1769074753.558342:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616616192 00000100:00100000:0.0:1769074753.558403:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13186 00000100:00100000:0.0:1769074753.558413:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62496:x1855007616616192:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074753.558456:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074753.558459:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1769074753.558486:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52125 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac55 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.558504:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.558509:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074753.558522:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90600/0x7fe2323adfd52133 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ac5c expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.558625:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90400/0x7fe2323adfd5213a lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.558651:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90400/0x7fe2323adfd5213a lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074753.558659:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90400/0x7fe2323adfd5213a lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074753.558670:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074753.558772:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90600/0x7fe2323adfd52133 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 13 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac5c expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.558786:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90400/0x7fe2323adfd5213a lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac5c expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.558794:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.558797:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074753.558799:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546f90400, rc 0) 00000100:00100000:0.0:1769074753.558884:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62496:x1855007616616192:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 472us (656us total) trans 8589943781 rc 0/0 00000100:00100000:0.0:1769074753.558892:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13186 00000100:00100000:0.0:1769074753.562664:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.562960:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13187 00000100:00100000:0.0:1769074753.562984:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62496:x1855007616616320:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074753.563008:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074753.563029:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90c00/0x7fe2323adfd52148 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ac63 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.563093:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90c00/0x7fe2323adfd52148 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac63 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.563107:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90c00/0x7fe2323adfd52148 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac63 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.563125:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.563128:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074753.563131:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546f90c00, rc 0) 00000100:00100000:0.0:1769074753.563186:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62496:x1855007616616320:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 216us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.563193:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13187 00000100:00100000:1.0:1769074753.571653:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.571700:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.571734:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.571765:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.571781:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.571798:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.571924:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616616448 00000100:00100000:0.0:1769074753.571936:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616616448 00000100:00100000:0.0:1769074753.572013:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616616576 00000100:00100000:0.0:1769074753.572017:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616616576 00000100:00100000:0.0:1769074753.572035:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616616704 00000100:00100000:0.0:1769074753.572037:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616616704 00000100:00100000:0.0:1769074753.572049:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616616832 00000100:00100000:0.0:1769074753.572052:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616616832 00000100:00100000:0.0:1769074753.572063:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616616960 00000100:00100000:0.0:1769074753.572065:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616616960 00000100:00100000:0.0:1769074753.572079:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617088 00000100:00100000:0.0:1769074753.572081:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617088 00000100:00100000:0.0:1769074753.572095:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13481 00000100:00100000:0.0:1769074753.572103:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616616448:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:0.0:1769074753.572150:0:8396:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546f90600/0x7fe2323adfd5214f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.572176:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 10 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.572185:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd520f4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac2b expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.572194:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 25 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.572202:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.572204:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.572207:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f90600/0x7fe2323adfd5214f lrc: 6/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.572248:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 25 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.572264:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 25 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.572281:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625029504:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074753.572386:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd520f4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac2b expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.572395:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd520f4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac2b expref: 16 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.572409:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625029632:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074753.572441:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 10 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.572449:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 10 pid: 27844 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.572459:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625029760:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074753.572480:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a4229c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074753.573578:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13484 00000100:00100000:1.0:1769074753.573590:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616616832:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:1.0:1769074753.573612:0:8399:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd52164 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.573638:0:8399:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.573649:0:8399:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52110 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac40 expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074753.573655:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074753.573657:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074753.573659:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd52164 lrc: 5/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.573684:0:8399:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52110 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac40 expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074753.573694:0:8399:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52110 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac40 expref: 16 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.573704:0:8399:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8399:1855007625030656:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.573759:0:8399:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.573768:0:8399:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.573777:0:8399:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8399:1855007625030784:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074753.573799:0:8399:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397f00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074753.598292:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.598319:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.598336:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.598354:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.598369:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.598390:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.598418:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617344 00000100:00100000:0.0:1769074753.598426:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617344 00010000:00010000:0.0:1769074753.598465:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd520f4 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac2b expref: 17 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.598490:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617216 00000100:00100000:0.0:1769074753.598493:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617216 00010000:00010000:0.0:1769074753.598502:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 26 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.598511:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617728 00000100:00100000:0.0:1769074753.598515:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617728 00010000:00010000:0.0:1769074753.598526:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd520bc lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac08 expref: 16 pid: 28954 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.598534:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617984 00000100:00100000:0.0:1769074753.598537:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617984 00010000:00010000:0.0:1769074753.598550:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abfa expref: 21 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.598560:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618368 00000100:00100000:0.0:1769074753.598562:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618368 00010000:00010000:0.0:1769074753.598572:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd52102 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac47 expref: 18 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.598580:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617856 00000100:00100000:0.0:1769074753.598583:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617856 00010000:00010000:0.0:1769074753.598613:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd520fb lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac39 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.598643:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6650 00000100:00100000:0.0:1769074753.598652:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2406:x1855007616617344:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.598666:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.598686:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd520f4 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac2b expref: 18 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.598698:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd520f4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac2b expref: 18 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.598726:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.598774:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2406:x1855007616617344:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 124us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.598781:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6650 00000100:00100000:0.0:1769074753.598799:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6651 00000100:00100000:0.0:1769074753.598831:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616617216:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.598839:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.598847:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 26 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.598857:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b546f91a00/0x7fe2323adfd52037 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abbb expref: 26 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.598869:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.598887:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616617216:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 82us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.598893:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6651 00000100:00100000:0.0:1769074753.598905:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6652 00000100:00100000:0.0:1769074753.598911:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616617728:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.598919:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.598931:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd520bc lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac08 expref: 16 pid: 28954 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.598940:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd520bc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac08 expref: 16 pid: 28954 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.598951:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.598969:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616617728:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 58us (631us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.600144:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6652 00000100:00100000:0.0:1769074753.600185:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6653 00000100:00100000:0.0:1769074753.600199:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616617984:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.600213:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.600232:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abfa expref: 21 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.600245:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abfa expref: 21 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.600271:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.600312:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616617984:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 119us (1956us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.600317:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6653 00000100:00100000:0.0:1769074753.600331:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6654 00000100:00100000:0.0:1769074753.600338:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2407:x1855007616618368:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.600347:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.600360:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd52102 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac47 expref: 17 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.600370:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd52102 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac47 expref: 17 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.600383:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.600401:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2407:x1855007616618368:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 63us (2030us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.600407:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6654 00000100:00100000:0.0:1769074753.600421:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6655 00000100:00100000:0.0:1769074753.600428:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616617856:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.600437:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.600445:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd520fb lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac39 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.600454:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd520fb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac39 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.600467:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.600470:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.600476:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.600486:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.600508:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.600527:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616617856:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 99us (2135us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.600533:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6655 00000100:00100000:0.0:1769074753.600643:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.600685:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618112 00000100:00100000:0.0:1769074753.600694:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618112 00010000:00010000:0.0:1769074753.600717:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52110 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac40 expref: 17 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.600734:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6656 00000100:00100000:0.0:1769074753.600740:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2407:x1855007616618112:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.600747:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.600756:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52110 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac40 expref: 17 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.600764:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52110 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac40 expref: 17 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.600776:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.600796:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2407:x1855007616618112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 56us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.600802:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6656 00000100:00100000:0.0:1769074753.600890:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.600904:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618496 00000100:00100000:0.0:1769074753.600909:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618496 00010000:00010000:0.0:1769074753.600921:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 11 pid: 27844 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.600935:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6657 00000100:00100000:0.0:1769074753.600940:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616618496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.600947:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.600955:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 11 pid: 27844 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.600968:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd52029 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abad expref: 11 pid: 27844 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.600992:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.600994:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.600997:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546f90600/0x7fe2323adfd5214f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.601003:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f90600/0x7fe2323adfd5214f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.601016:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.601037:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616618496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 96us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.601043:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6657 00000100:00100000:0.0:1769074753.601083:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.601096:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617472 00000100:00100000:0.0:1769074753.601099:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617472 00010000:00010000:0.0:1769074753.601112:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abde expref: 21 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.601126:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6658 00000100:00100000:0.0:1769074753.601131:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616617472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.601137:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.601146:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abde expref: 21 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.601159:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abde expref: 21 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.601170:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b546f90e00/0x7fe2323adfd52092 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abde expref: 21 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.601182:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.601199:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616617472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 68us (116us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.601205:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6658 00000100:00100000:0.0:1769074753.601243:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.601256:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618240 00000100:00100000:0.0:1769074753.601259:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618240 00010000:00010000:0.0:1769074753.601272:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 16 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.601286:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6659 00000100:00100000:0.0:1769074753.601292:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616618240:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.601298:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.601306:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 16 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.601316:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 16 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.601332:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd5203e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abc2 expref: 16 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.601341:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.601357:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616618240:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 66us (114us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.601362:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6659 00000100:00100000:0.0:1769074753.601420:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.601437:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616617600 00000100:00100000:0.0:1769074753.601441:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616617600 00010000:00010000:0.0:1769074753.601454:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd52109 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac32 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.601466:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6660 00000100:00100000:0.0:1769074753.601471:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616617600:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.601477:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.601486:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd52109 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac32 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.601493:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd52109 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac32 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.601504:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd52109 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac32 expref: 14 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.601518:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.601520:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.601523:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd52156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.601529:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd52156 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.601549:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.601565:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616617600:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 94us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.601571:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6660 00010000:00010000:1.0:1769074753.604929:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd520bc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac08 expref: 14 pid: 28954 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.604941:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa22680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625030272:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.604952:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd520fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac39 expref: 11 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.604959:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625030400:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.604986:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd520ae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abfa expref: 19 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.604996:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625030528:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.605019:0:36516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605028:0:36516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605035:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605052:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605060:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605067:0:36516:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605074:0:36516:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b6725b4000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074753.605077:0:36516:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605083:0:36516:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605095:0:36516:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.605101:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd5215d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074753.605145:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616616704:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 32140us (33410us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.605153:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13483 00000100:00100000:0.0:1769074753.627747:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.627779:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.627796:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.627837:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.627852:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.627872:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.627893:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618624 00000100:00100000:0.0:1769074753.627901:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618624 00010000:00010000:0.0:1769074753.627939:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5201b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aba6 expref: 10 pid: 8396 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.627966:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618752 00000100:00100000:0.0:1769074753.627968:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618752 00010000:00010000:0.0:1769074753.627994:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 25 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.628013:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616618880 00000100:00100000:0.0:1769074753.628016:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616618880 00010000:00010000:0.0:1769074753.628030:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd52117 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac4e expref: 10 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:0.0:1769074753.628042:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616619008 00000100:00100000:0.0:1769074753.628044:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616619008 00010000:00010000:0.0:1769074753.628055:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd5208b lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd0 expref: 14 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.628066:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616619136 00000100:00100000:0.0:1769074753.628068:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616619136 00010000:00010000:0.0:1769074753.628080:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd52006 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab8a expref: 26 pid: 36516 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.628089:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616619264 00000100:00100000:0.0:1769074753.628091:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616619264 00010000:00010000:0.0:1769074753.628100:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 19 pid: 27843 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.628113:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6661 00000100:00100000:0.0:1769074753.628120:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616618624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.628135:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.628157:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5201b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aba6 expref: 10 pid: 8396 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.628175:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5201b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aba6 expref: 10 pid: 8396 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.628199:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5201b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aba6 expref: 10 pid: 8396 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.628219:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.628269:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616618624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 148us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.628276:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6661 00000100:00100000:0.0:1769074753.628293:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6662 00000100:00100000:0.0:1769074753.628298:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616618752:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.628305:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.628313:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 26 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628323:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 26 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628335:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd520d1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac1d expref: 26 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628345:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.628347:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.628351:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd52164 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.628361:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd52164 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.628384:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.628398:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616618752:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 100us (619us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.628404:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6662 00000100:00100000:0.0:1769074753.628433:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6663 00000100:00100000:0.0:1769074753.628438:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616618880:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.628444:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.628455:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd52117 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac4e expref: 10 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.628464:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd52117 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac4e expref: 10 pid: 27844 timeout: 1841 lvb_type: 1 00010000:00010000:0.0:1769074753.628476:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.628488:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616618880:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 50us (692us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.628494:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6663 00000100:00100000:0.0:1769074753.628503:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6664 00000100:00100000:0.0:1769074753.628508:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616619008:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.628514:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.628519:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd5208b lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd0 expref: 14 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628529:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd5208b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd0 expref: 14 pid: 28954 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628539:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.628550:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616619008:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 42us (713us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.628555:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6664 00000100:00100000:0.0:1769074753.628564:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6665 00000100:00100000:0.0:1769074753.628569:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616619136:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.628574:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.628580:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd52006 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab8a expref: 24 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628590:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd52006 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab8a expref: 24 pid: 36516 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628599:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.628601:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.628604:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5216b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.628612:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5216b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.628621:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.628649:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616619136:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 80us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.628656:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6665 00000100:00100000:0.0:1769074753.628666:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6666 00000100:00100000:0.0:1769074753.628672:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616619264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.628677:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.628684:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 19 pid: 27843 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628694:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 19 pid: 27843 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.628705:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.628707:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.628710:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.628718:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074753.628727:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.628742:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616619264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 70us (869us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.628748:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6666 00010000:00010000:1.0:1769074753.629024:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd5208b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879abd0 expref: 13 pid: 28954 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.629037:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625031296:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.629049:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd52117 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac4e expref: 9 pid: 27844 timeout: 1841 lvb_type: 1 00000100:00100000:1.0:1769074753.629057:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625031424:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.629066:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52014 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ab9f expref: 18 pid: 27843 timeout: 1841 lvb_type: 0 00000100:00100000:1.0:1769074753.629074:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625031552:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074753.629095:0:28954:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629109:0:28954:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629116:0:28954:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629133:0:28954:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629141:0:28954:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629149:0:28954:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629155:0:28954:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f71400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074753.629159:0:28954:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629166:0:28954:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629179:0:28954:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074753.629186:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52172 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074753.629243:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2409:x1855007616617088:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 54526us (57443us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.629251:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13486 00000100:00100000:0.0:1769074753.643133:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.650145:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.655968:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.656017:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.656052:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.656074:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.656090:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.656147:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.656180:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616620160 00000100:00100000:0.0:1769074753.656188:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616620160 00010000:00010000:0.0:1769074753.656227:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70400/0x7fe2323adfd516c8 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5d3 expref: 229 pid: 10283 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.656263:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616620288 00000100:00100000:0.0:1769074753.656266:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616620288 00010000:00010000:0.0:1769074753.656280:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cdebe00/0x7fe2323adfd516c1 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5cc expref: 13 pid: 6513 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.656292:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616620416 00000100:00100000:0.0:1769074753.656295:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616620416 00010000:00010000:0.0:1769074753.656305:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773c800/0x7fe2323adfd516ac lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5b7 expref: 14 pid: 8855 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.656314:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616620544 00000100:00100000:0.0:1769074753.656315:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616620544 00010000:00010000:0.0:1769074753.656324:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e8c lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaaa expref: 230 pid: 6512 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.656333:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616620672 00000100:00100000:0.0:1769074753.656336:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616620672 00010000:00010000:0.0:1769074753.656345:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f91400/0x7fe2323adfd51b67 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a8b2 expref: 15 pid: 6514 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.656358:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6667 00000100:00100000:0.0:1769074753.656365:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:2409:x1855007616620160:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.656379:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.656385:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70400/0x7fe2323adfd516c8 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5d3 expref: 230 pid: 10283 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656396:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70400/0x7fe2323adfd516c8 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5d3 expref: 230 pid: 10283 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656419:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.656425:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.656434:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.656494:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:2409:x1855007616620160:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 129us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.656502:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6667 00000100:00100000:0.0:1769074753.656518:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6668 00000100:00100000:0.0:1769074753.656524:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616620288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.656531:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.656535:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cdebe00/0x7fe2323adfd516c1 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5cc expref: 15 pid: 6513 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656543:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cdebe00/0x7fe2323adfd516c1 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5cc expref: 15 pid: 6513 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656554:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.656557:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.656566:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.656582:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616620288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 59us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.656588:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6668 00000100:00100000:0.0:1769074753.656601:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6669 00000100:00100000:0.0:1769074753.656607:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616620416:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.656614:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.656619:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773c800/0x7fe2323adfd516ac lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5b7 expref: 14 pid: 8855 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656639:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773c800/0x7fe2323adfd516ac lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a5b7 expref: 14 pid: 8855 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656650:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.656653:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.656658:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.656677:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616620416:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 71us (603us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.656683:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6669 00000100:00100000:0.0:1769074753.656715:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6670 00000100:00100000:0.0:1769074753.656721:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:2408:x1855007616620544:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.656727:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.656732:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e8c lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaaa expref: 229 pid: 6512 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656745:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd51e8c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aaaa expref: 229 pid: 6512 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656755:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.656759:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.656765:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.656782:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:2408:x1855007616620544:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 61us (690us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.656788:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6670 00000100:00100000:0.0:1769074753.656798:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6671 00000100:00100000:0.0:1769074753.656841:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616620672:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.656849:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.656853:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f91400/0x7fe2323adfd51b67 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a8b2 expref: 13 pid: 6514 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656862:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f91400/0x7fe2323adfd51b67 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a8b2 expref: 13 pid: 6514 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.656872:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.656876:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.656882:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.656905:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616620672:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 100us (758us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.656914:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6671 00000100:00100000:0.0:1769074753.661251:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.664510:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.664606:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616620928 00000100:00100000:0.0:1769074753.664618:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616620928 00010000:00010000:0.0:1769074753.664672:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd51754 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a63c expref: 228 pid: 8855 timeout: 1841 lvb_type: 0 00000100:00100000:0.0:1769074753.664711:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6673 00000100:00100000:0.0:1769074753.664721:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:2409:x1855007616620928:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074753.664738:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074753.664745:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd51754 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a63c expref: 228 pid: 8855 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.664759:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd51754 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a63c expref: 228 pid: 8855 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.664784:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd51754 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a63c expref: 228 pid: 8855 timeout: 1841 lvb_type: 0 00010000:00010000:0.0:1769074753.664799:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074753.664842:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.664854:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.664860:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52179 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074753.664900:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074753.664956:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2409:x1855007616620928:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 235us (456us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.664963:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6673 00000100:00100000:0.0:1769074753.673267:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625033984:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.673310:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074753.673332:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625034496:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.673350:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074753.673368:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1769074753.673374:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625033856:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.673393:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074753.673411:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625034368:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.673428:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.673551:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625033600:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.673751:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625033728:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.674818:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.674854:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625034112:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.674880:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.674918:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625033600 00000100:00100000:1.0:1769074753.674923:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625033600 00000100:00100000:1.0:1769074753.674950:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625034112 00000100:00100000:1.0:1769074753.674953:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625034112 00000100:00100000:1.0:1769074753.674965:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13496 00000100:00100000:1.0:1769074753.674985:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625034496:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074753.675063:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb size from disk: 8388608 -> 8388608 00002000:00010000:1.0:1769074753.675066:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.675068:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074753.675071:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.675073:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb blocks from disk: 8192 -> 8192 00000100:00100000:1.0:1769074753.675109:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625034496:12345-0@lo:2:osp-syn-1-0.0 Request processed in 137us (1758us total) trans 17179874414 rc 0/0 00000100:00100000:1.0:1769074753.675116:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13496 00000100:00100000:1.0:1769074753.675132:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13497 00000100:00100000:1.0:1769074753.675139:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625033856:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.675209:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625033856:12345-0@lo:2:osp-syn-1-0.0 Request processed in 71us (1815us total) trans 17179874415 rc 0/0 00000100:00100000:1.0:1769074753.675216:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13497 00000100:00100000:1.0:1769074753.675228:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13498 00000100:00100000:1.0:1769074753.675234:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625034368:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074753.675282:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074753.675285:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.675287:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074753.675289:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.675290:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:1.0:1769074753.675315:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625034368:12345-0@lo:2:osp-syn-1-0.0 Request processed in 82us (1886us total) trans 17179874416 rc 0/0 00000100:00100000:1.0:1769074753.675320:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13498 00000100:00100000:1.0:1769074753.675332:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13499 00000100:00100000:1.0:1769074753.675337:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625033600:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.675406:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625033600:12345-0@lo:2:osp-syn-1-0.0 Request processed in 69us (598us total) trans 17179874417 rc 0/0 00000100:00100000:1.0:1769074753.675412:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13499 00000100:00100000:1.0:1769074753.675425:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13500 00000100:00100000:1.0:1769074753.675431:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625034112:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074753.675719:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074753.675723:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074745 -> 1769074745 00002000:00010000:1.0:1769074753.675725:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074753.675726:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074745 -> 1769074745 00002000:00010000:1.0:1769074753.675728:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074753.675755:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625034112:12345-0@lo:2:osp-syn-1-0.0 Request processed in 324us (874us total) trans 17179874418 rc 0/0 00000100:00100000:1.0:1769074753.675762:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13500 00000004:00080000:1.0:1769074753.675832:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54538b100/2, rc 0, transno 17179874415 00000100:00100000:1.0:1769074753.675846:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625033856:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074753.675858:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54ce9d500/2, rc 0, transno 17179874416 00000100:00100000:1.0:1769074753.675861:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625034368:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074753.675869:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54ce9d180/2, rc 0, transno 17179874414 00000100:00100000:1.0:1769074753.675872:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625034496:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.675928:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625034624:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.675959:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1769074753.676116:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545389880/2, rc 0, transno 17179874417 00000100:00100000:1.0:1769074753.676121:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625033600:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074753.676129:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545388000/2, rc 0, transno 17179874418 00000100:00100000:1.0:1769074753.676132:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625034112:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.676150:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625034624 00000100:00100000:1.0:1769074753.676155:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625034624 00000100:00100000:1.0:1769074753.676180:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13501 00000100:00100000:1.0:1769074753.676185:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625034624:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074753.676251:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074753.676255:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.676257:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074753.676259:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.676261:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074753.676291:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625034624:12345-0@lo:2:osp-syn-1-0.0 Request processed in 105us (331us total) trans 17179874419 rc 0/0 00000100:00100000:1.0:1769074753.676297:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13501 00000004:00080000:1.0:1769074753.676322:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54475ad80/2, rc 0, transno 17179874419 00000100:00100000:1.0:1769074753.676326:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625034624:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.676472:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074753.676501:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625034240:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074753.676522:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.676843:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625033728 00000100:00100000:1.0:1769074753.676849:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625033728 00000100:00100000:1.0:1769074753.676871:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625034240 00000100:00100000:1.0:1769074753.676874:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625034240 00000100:00100000:1.0:1769074753.676891:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13502 00000100:00100000:1.0:1769074753.676896:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625033728:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.677005:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+15:3650:x1855007625033728:12345-0@lo:2:osp-syn-1-0.0 Request processed in 108us (533us total) trans 17179874420 rc 0/0 00000100:00100000:1.0:1769074753.677012:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13502 00000100:00100000:1.0:1769074753.677026:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13503 00000100:00100000:1.0:1769074753.677032:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625034240:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074753.677086:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074753.677089:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.677092:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074753.677094:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074746 -> 1769074746 00002000:00010000:1.0:1769074753.677096:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074753.677121:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+15:3650:x1855007625034240:12345-0@lo:2:osp-syn-1-0.0 Request processed in 89us (598us total) trans 17179874421 rc 0/0 00000100:00100000:1.0:1769074753.677127:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13503 00000004:00080000:0.0:1769074753.680255:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54538a300/2, rc 0, transno 17179874420 00000100:00100000:0.0:1769074753.680264:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625033728:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074753.680275:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545389f80/2, rc 0, transno 17179874421 00000100:00100000:0.0:1769074753.680278:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625034240:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074753.681257:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54538b800/2, rc 0, transno 17179874413 00000100:00100000:0.0:1769074753.681263:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625033984:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074753.681348:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074753.687747:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.693367:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.693404:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.693439:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.693550:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074753.693579:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616621312 00000100:00100000:0.0:1769074753.693588:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616621312 00000100:00100000:0.0:1769074753.693644:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616621440 00000100:00100000:0.0:1769074753.693648:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616621440 00000100:00100000:0.0:1769074753.693665:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616621568 00000100:00100000:0.0:1769074753.693667:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616621568 00000100:00100000:0.0:1769074753.693684:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616621696 00000100:00100000:0.0:1769074753.693686:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616621696 00000100:00100000:0.0:1769074753.693705:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13504 00000100:00100000:0.0:1769074753.693713:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616621312:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074753.693732:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074753.693760:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ac78 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074753.693786:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bc:0x0] initial LVB size: 1048576, mtime: 0x6971f040, atime: 0x6971f03e, ctime: 0x6971f040, blocks: 0x7e8 00010000:00010000:0.0:1769074753.693834:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.693837:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074753.693851:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac78 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.693873:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac78 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074753.693888:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546f91800, rc 0) 00000100:00100000:0.0:1769074753.693959:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616621312:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 246us (603us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074753.693967:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13504 00000100:00100000:0.0:1769074753.952988:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074753.953012:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625034880:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074753.953078:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074753.953140:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625034752:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074753.953165:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074753.953352:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625034880 00000100:00100000:1.0:1769074753.953359:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625034880 00000100:00100000:1.0:1769074753.953404:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625034752 00000100:00100000:1.0:1769074753.953407:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625034752 00000100:00100000:1.0:1769074753.953433:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1148 00000100:00100000:1.0:1769074753.953441:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625034880:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074753.953462:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 914974016 pending 2617344 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074753.953509:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625034880:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (431us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074753.953516:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1148 00000100:00100000:1.0:1769074753.953576:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625034752:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074753.953899:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625034880:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074753.953965:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625035008:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074754.465413:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625035136:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074754.814113:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.814159:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.814194:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.814221:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.814245:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616622080 00000100:00100000:1.0:1769074754.814256:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616622080 00000100:00100000:1.0:1769074754.814310:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616622336 00000100:00100000:1.0:1769074754.814313:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616622336 00000100:00100000:1.0:1769074754.814332:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616622592 00000100:00100000:1.0:1769074754.814334:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616622592 00000100:00100000:1.0:1769074754.814349:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616622208 00000100:00100000:1.0:1769074754.814352:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616622208 00000100:00100000:1.0:1769074754.814372:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13510 00000100:00100000:1.0:1769074754.814381:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616622080:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074754.814491:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13511 00000100:00100000:0.0:1769074754.814502:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616622336:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074754.814522:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616622080:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 139us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.814530:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13510 00000100:00100000:1.0:1769074754.814555:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13512 00000100:00100000:0.0:1769074754.814556:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616622336:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 53us (393us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.814565:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616622592:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074754.814570:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13511 00000100:00100000:0.0:1769074754.814591:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13513 00000100:00100000:0.0:1769074754.814600:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b358050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616622208:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074754.814623:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616622592:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 59us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.814630:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13512 00000100:00080000:1.0:1769074754.814651:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074754.814655:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074754.814661:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b358050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616622208:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 59us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074754.814669:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13513 00000100:00100000:1.0:1769074754.814704:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.814777:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074754.814792:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616622464 00000100:00100000:0.0:1769074754.814798:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616622464 00000100:00100000:1.0:1769074754.814798:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616622720 00000100:00100000:1.0:1769074754.814837:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616622720 00000100:00100000:0.0:1769074754.814855:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13514 00000100:00100000:0.0:1769074754.814864:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616622464:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074754.814912:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616622464:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 48us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.815385:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13515 00000100:00100000:1.0:1769074754.815395:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616622720:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074754.815438:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616622720:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 45us (661us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.815444:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13515 00000100:00100000:0.0:1769074754.818648:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13514 00000100:00100000:1.0:1769074754.848414:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616623744 00000100:00100000:1.0:1769074754.848422:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616623744 00010000:00010000:1.0:1769074754.848458:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd515cc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a516 expref: 13 pid: 27844 timeout: 1842 lvb_type: 0 00000100:00100000:1.0:1769074754.848490:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616623872 00000100:00100000:1.0:1769074754.848492:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616623872 00010000:00010000:1.0:1769074754.848513:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd515d3 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a51d expref: 18 pid: 27844 timeout: 1842 lvb_type: 0 00000100:00100000:1.0:1769074754.848531:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6674 00000100:00100000:1.0:1769074754.848538:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616623744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074754.848553:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074754.848571:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd515cc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a516 expref: 13 pid: 27844 timeout: 1842 lvb_type: 0 00010000:00010000:1.0:1769074754.848583:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd515cc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a516 expref: 13 pid: 27844 timeout: 1842 lvb_type: 0 00010000:00010000:1.0:1769074754.848617:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd515cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a516 expref: 13 pid: 27844 timeout: 1842 lvb_type: 0 00010000:00010000:1.0:1769074754.848634:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074754.848680:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616623744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 142us (2828us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.848687:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6674 00000100:00100000:1.0:1769074754.848703:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6675 00000100:00100000:1.0:1769074754.848709:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616623872:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074754.848717:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074754.848725:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd515d3 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a51d expref: 18 pid: 27844 timeout: 1842 lvb_type: 0 00010000:00010000:1.0:1769074754.848735:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd515d3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a51d expref: 18 pid: 27844 timeout: 1842 lvb_type: 0 00010000:00010000:1.0:1769074754.848748:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.848750:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.848756:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd521bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aca2 expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.848764:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd521bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aca2 expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.848798:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd521bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aca2 expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.848849:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656dbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625035520:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074754.848903:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00080000:1.0:1769074754.848920:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074754.856673:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656dbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625035520:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074754.856708:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074754.856798:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616623872:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8087us (8537us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074754.856832:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6675 00000100:00100000:1.0:1769074754.884364:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616624256 00000100:00100000:1.0:1769074754.884382:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616624256 00000100:00100000:1.0:1769074754.884459:0:28444:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616624384 00000100:00100000:1.0:1769074754.884463:0:28444:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616624384 00000100:00100000:1.0:1769074754.884835:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8211 00000100:00100000:1.0:1769074754.884847:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616624256:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074754.884891:0:28444:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.884909:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.884912:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.884917:0:28444:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.884948:0:28444:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.886671:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074754.911165:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8214 00000100:00100000:0.0:1769074754.911186:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2406:x1855007616624640:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:0.0:1769074754.911230:0:24027:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.911245:0:24027:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074754.911247:0:24027:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074754.911253:0:24027:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.911267:0:24027:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.928550:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.928583:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.928615:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.928643:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616624896 00000100:00100000:1.0:1769074754.928653:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616624896 00000100:00100000:1.0:1769074754.928712:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616624768 00000100:00100000:1.0:1769074754.928715:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616624768 00000100:00100000:1.0:1769074754.928739:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616625024 00000100:00100000:1.0:1769074754.928742:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616625024 00000100:00100000:1.0:1769074754.928765:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8215 00000100:00100000:1.0:1769074754.928773:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2406:x1855007616624896:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074754.928835:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.928851:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.928853:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.928859:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.928875:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.929409:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8216 00000100:00100000:1.0:1769074754.929420:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+20:2406:x1855007616624768:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074754.929442:0:8402:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.929451:0:8402:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.929452:0:8402:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.929456:0:8402:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.929465:0:8402:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.930060:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8217 00000100:00100000:1.0:1769074754.930071:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+21:2406:x1855007616625024:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074754.930090:0:24030:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd521f0 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24030 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.930099:0:24030:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.930100:0:24030:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.930104:0:24030:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd521f0 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24030 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.930114:0:24030:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd521f0 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24030 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.930722:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074754.930753:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616625152 00000100:00100000:1.0:1769074754.930760:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616625152 00000100:00100000:1.0:1769074754.930832:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8218 00000100:00100000:1.0:1769074754.930840:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2406:x1855007616625152:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074754.930862:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.930872:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.930873:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074754.930877:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.930887:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074754.934379:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074754.951999:0:28444:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.952021:0:28444:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.952029:0:28444:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.952036:0:28444:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910ce00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074754.952041:0:28444:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.952051:0:28444:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.952071:0:28444:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.952077:0:28444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd521c6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.952720:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+23:2406:x1855007616624256:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 67867us (69526us total) trans 0 rc 1036288/1036288 00000100:00100000:1.0:1769074754.952736:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8211 00010000:00010000:0.0:1769074754.973714:0:24027:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.973735:0:24027:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.973744:0:24027:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.973752:0:24027:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b546f91c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074754.973755:0:24027:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.973768:0:24027:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.973785:0:24027:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074754.973792:0:24027:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546f91c00/0x7fe2323adfd521db lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24027 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074754.973915:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+21:2406:x1855007616624640:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 62731us (66577us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074754.973925:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8214 00010000:00010000:1.0:1769074754.974362:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.974373:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.974380:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.974386:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074754.974389:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.974396:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.974408:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.974415:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd521d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.974455:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2406:x1855007616624512:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 66974us (87792us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074754.974463:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8213 00010000:00010000:1.0:1769074754.984127:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.984148:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.984157:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.984164:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549e46800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074754.984168:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.984177:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.984198:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.984210:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd521e2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.984329:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616624896:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 55551us (55784us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074754.984340:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8215 00010000:00010000:1.0:1769074754.997136:0:8402:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.997156:0:8402:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.997165:0:8402:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.997171:0:8402:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549e47c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074754.997175:0:8402:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.997183:0:8402:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.997200:0:8402:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074754.997206:0:8402:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd521e9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074754.997308:0:8402:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2406:x1855007616624768:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 67885us (68719us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074754.997320:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8216 00000100:00100000:0.0:1769074755.011944:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012003:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012038:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616625664 00000100:00100000:0.0:1769074755.012048:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616625664 00000100:00100000:0.0:1769074755.012121:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2275 00000100:00100000:0.0:1769074755.012130:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616625664:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.012209:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616625664:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 79us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.012218:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2275 00000100:00080000:0.0:1769074755.012244:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074755.012277:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012394:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012409:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616625536 00000100:00100000:0.0:1769074755.012414:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616625536 00000100:00100000:0.0:1769074755.012443:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616625920 00000100:00100000:0.0:1769074755.012447:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616625920 00000100:00100000:0.0:1769074755.012468:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2276 00000100:00100000:0.0:1769074755.012474:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+220:2409:x1855007616625536:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.012528:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+220:2409:x1855007616625536:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 53us (249us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.012535:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2276 00000100:00100000:0.0:1769074755.012548:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2277 00000100:00100000:0.0:1769074755.012554:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616625920:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.012581:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616625920:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.012586:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2277 00000100:00080000:0.0:1769074755.012619:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074755.012645:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012688:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012735:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012753:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.012766:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616626048 00000100:00100000:0.0:1769074755.012771:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616626048 00000100:00100000:0.0:1769074755.012796:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616626432 00000100:00100000:0.0:1769074755.012799:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616626432 00000100:00100000:0.0:1769074755.012869:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2278 00000100:00100000:0.0:1769074755.012875:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2409:x1855007616626048:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.012904:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2409:x1855007616626048:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.012911:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2278 00000100:00100000:0.0:1769074755.012925:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2279 00000100:00100000:0.0:1769074755.012933:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2408:x1855007616626432:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.012956:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2408:x1855007616626432:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.012961:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2279 00000100:00080000:0.0:1769074755.012976:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074755.012986:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616625792 00000100:00100000:0.0:1769074755.012990:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616625792 00000100:00100000:0.0:1769074755.013016:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616626176 00000100:00100000:0.0:1769074755.013019:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616626176 00000100:00100000:0.0:1769074755.013038:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1425 00000100:00100000:0.0:1769074755.013044:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616625792:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.013070:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616625792:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.013076:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1425 00000100:00100000:0.0:1769074755.013088:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1426 00000100:00100000:0.0:1769074755.013094:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616626176:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.013116:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616626176:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 23us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.013122:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1426 00000100:00080000:0.0:1769074755.013136:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074755.013166:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.013231:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616626304 00000100:00100000:0.0:1769074755.013235:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616626304 00000100:00100000:0.0:1769074755.013259:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2280 00000100:00100000:0.0:1769074755.013265:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616626304:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074755.013296:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616626304:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 31us (130us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.013302:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2280 00000100:00080000:0.0:1769074755.013315:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1769074755.013433:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.013446:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.013456:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.013463:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549e46600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074755.013468:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.013478:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.013497:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.013504:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd521f7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074755.013544:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616625152:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 82705us (82824us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074755.013557:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8218 00000100:00100000:1.0:1769074755.020176:0:24030:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2406:x1855007616625024:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 90095us (91556us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074755.020191:0:24030:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8217 00000100:00100000:0.0:1769074755.039169:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.053236:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.054756:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616626688 00000100:00100000:1.0:1769074755.054773:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616626688 00000001:00080000:0.0:1769074755.094385:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841104 is committed 00000001:00080000:0.0:1769074755.094408:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841102 is committed 00000001:00080000:0.0:1769074755.094435:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841094 is committed 00000001:00080000:0.0:1769074755.094441:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841093 is committed 00002000:00010000:0.0:1769074755.094946:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074745 -> 1769074754 00002000:00010000:0.0:1769074755.095026:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074745 -> 1769074754 00010000:00010000:0.0:1769074755.095043:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.095055:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.095064:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.095071:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074755.095075:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.095083:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.095101:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.095113:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88000/0x7fe2323adfd52205 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074755.095271:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616626560:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 55872us (56106us total) trans 21474841104 rc 0/0 00000100:00100000:0.0:1769074755.095281:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8220 00000100:00100000:1.0:1769074755.107298:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.107339:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.107506:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.111147:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.116161:0:8402:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616627200 00000100:00100000:1.0:1769074755.116175:0:8402:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616627200 00000100:00100000:1.0:1769074755.116238:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8224 00000100:00100000:1.0:1769074755.116247:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616627200:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074755.116280:0:8402:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.116296:0:8402:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074755.116299:0:8402:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074755.116302:0:8402:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.116312:0:8402:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074755.118414:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.118784:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.120797:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.120893:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616627328 00000100:00100000:0.0:1769074755.120904:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616627328 00010000:00010000:0.0:1769074755.120962:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd514ec lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4a6 expref: 17 pid: 28519 timeout: 1842 lvb_type: 0 00000100:00100000:0.0:1769074755.121015:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616627584 00000100:00100000:0.0:1769074755.121019:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616627584 00010000:00010000:0.0:1769074755.121050:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd515da lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a524 expref: 6 pid: 36516 timeout: 1842 lvb_type: 0 00000100:00100000:0.0:1769074755.121080:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6677 00000100:00100000:0.0:1769074755.121090:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2409:x1855007616627328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074755.121109:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074755.121138:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd514ec lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4a6 expref: 17 pid: 28519 timeout: 1842 lvb_type: 0 00010000:00010000:0.0:1769074755.121152:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd514ec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4a6 expref: 17 pid: 28519 timeout: 1842 lvb_type: 0 00010000:00010000:0.0:1769074755.121178:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd514ec lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4a6 expref: 17 pid: 28519 timeout: 1842 lvb_type: 0 00010000:00010000:0.0:1769074755.121196:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.121198:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.121204:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.121213:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.121223:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.121225:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.121228:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.121235:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.121269:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074755.121337:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616627328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 248us (2930us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.121346:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6677 00000100:00100000:0.0:1769074755.121370:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6678 00000100:00100000:0.0:1769074755.121377:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+6:2409:x1855007616627584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074755.121386:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074755.121399:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd515da lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a524 expref: 6 pid: 36516 timeout: 1842 lvb_type: 0 00010000:00010000:0.0:1769074755.121408:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd515da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a524 expref: 6 pid: 36516 timeout: 1842 lvb_type: 0 00010000:00010000:0.0:1769074755.121439:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd515da lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a524 expref: 6 pid: 36516 timeout: 1842 lvb_type: 0 00010000:00010000:0.0:1769074755.121453:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074755.121473:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2409:x1855007616627584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 97us (2689us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.121480:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6678 00000100:00080000:0.0:1769074755.122044:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1769074755.122705:0:8402:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.122718:0:8402:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd52221 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1769074755.176461:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841105 is committed 00002000:00010000:1.0:1769074755.182495:0:16436:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074745 -> 1769074754 00002000:00010000:1.0:1769074755.182506:0:16436:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074745 -> 1769074754 00010000:00010000:1.0:1769074755.182521:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.182533:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.182541:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.182548:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074755.182552:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.182560:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.182585:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.182592:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52213 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074755.182704:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616627072:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 70354us (75359us total) trans 21474841105 rc 0/0 00000100:00100000:1.0:1769074755.182715:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8222 00000001:00080000:0.0:1769074755.185774:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841106 is committed 00010000:00010000:1.0:1769074755.186774:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.186785:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.186791:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.186797:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074755.186800:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.186825:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.186837:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.186848:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5220c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074755.186928:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616626944:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 75004us (79635us total) trans 17179874424 rc 0/0 00000100:00100000:1.0:1769074755.186939:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8221 00000100:00100000:1.0:1769074755.199502:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.204096:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616627840 00000100:00100000:1.0:1769074755.204106:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616627840 00000100:00100000:1.0:1769074755.204196:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8226 00000100:00100000:1.0:1769074755.204207:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616627840:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074755.204250:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5222f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.204269:0:16437:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd514fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4b4 expref: 13 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.204278:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074755.204281:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074755.204284:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5222f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.204322:0:16437:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd514fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4b4 expref: 13 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.204336:0:16437:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd514fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4b4 expref: 13 pid: 28519 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074755.204355:0:16437:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625036288:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074755.204697:0:16437:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074755.211843:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616627968 00000100:00100000:0.0:1769074755.211861:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616627968 00010000:00010000:1.0:1769074755.212072:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd514f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4ad expref: 12 pid: 28954 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074755.212179:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6680 00000100:00100000:1.0:1769074755.212188:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616627968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074755.212204:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074755.212225:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd514f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4ad expref: 12 pid: 28954 timeout: 1843 lvb_type: 0 00010000:00010000:1.0:1769074755.212238:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd514f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4ad expref: 12 pid: 28954 timeout: 1843 lvb_type: 0 00010000:00010000:1.0:1769074755.212261:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd514f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4ad expref: 12 pid: 28954 timeout: 1843 lvb_type: 0 00010000:00010000:1.0:1769074755.212279:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074755.212281:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074755.212285:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24030 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.212295:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52228 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24030 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.212339:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074755.212405:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616627968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 218us (4276us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074755.212413:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6680 00000100:00100000:1.0:1769074755.218590:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074755.219931:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5222f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.219942:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5222f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1769074755.258211:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841107 is committed 00000100:00100000:1.0:1769074755.271345:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.277392:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074755.279239:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.279254:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1769074755.300683:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841108 is committed 00000100:00100000:1.0:1769074755.313565:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074755.319736:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031e00/0x7fe2323adfd5151d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4c9 expref: 12 pid: 28519 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074755.319760:0:16437:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625036544:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074755.319784:0:16437:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.319799:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.319859:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074755.373222:0:16437:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074746 -> 1769074754 00002000:00010000:1.0:1769074755.373228:0:16437:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074746 -> 1769074754 00010000:00010000:1.0:1769074755.373248:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.373259:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.373267:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.373274:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074755.373278:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.373285:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.373304:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.373316:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5223d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074755.373569:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616628480:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 59665us (60009us total) trans 17179874426 rc 0/0 00000100:00100000:1.0:1769074755.373581:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8228 00000100:00100000:1.0:1769074755.387486:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616628736 00000100:00100000:1.0:1769074755.387497:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616628736 00000100:00100000:1.0:1769074755.387572:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8229 00000100:00100000:1.0:1769074755.387583:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616628736:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074755.387633:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.387655:0:16437:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd51524 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4d0 expref: 11 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.387664:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074755.387665:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074755.387670:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.387707:0:16437:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd51524 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4d0 expref: 11 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.387722:0:16437:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd51524 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4d0 expref: 11 pid: 28954 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074755.387741:0:16437:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625036672:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074755.387893:0:16437:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074755.391287:0:16437:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625036672:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074755.391319:0:16437:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.394831:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.394841:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1769074755.443519:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874427 is committed 00002000:00010000:1.0:1769074755.443739:0:16437:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074746 -> 1769074755 00002000:00010000:1.0:1769074755.443744:0:16437:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074746 -> 1769074755 00010000:00010000:1.0:1769074755.444154:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.444166:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.444176:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.444183:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549e47e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074755.444188:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.444197:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.444218:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.444227:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52244 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074755.444442:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616628736:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 56854us (57070us total) trans 17179874427 rc 0/0 00000100:00100000:1.0:1769074755.444453:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8229 00000100:00100000:1.0:1769074755.455242:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074755.462834:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5152b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879a4d7 expref: 10 pid: 28954 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074755.462860:0:16437:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625036800:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074755.462888:0:16437:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.462900:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.462907:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074755.475398:0:16437:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4808704 00002000:00010000:1.0:1769074755.475404:0:16437:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074746 -> 1769074755 00002000:00010000:1.0:1769074755.475407:0:16437:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074746 -> 1769074755 00010000:00010000:1.0:1769074755.475571:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.475584:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.475592:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.475609:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074755.475613:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.475624:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.475647:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.475655:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5224b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074755.476175:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616628992:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 20577us (20937us total) trans 17179874428 rc 0/0 00000100:00100000:1.0:1769074755.476189:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8230 00000100:00100000:0.0:1769074755.508803:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616629376 00000100:00100000:0.0:1769074755.508865:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616629376 00000100:00100000:0.0:1769074755.508955:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13191 00000100:00100000:0.0:1769074755.508967:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62496:x1855007616629376:12345-192.168.204.60@tcp:36:mirror_io.0 00010000:00010000:0.0:1769074755.509025:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074755.509059:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70400/0x7fe2323adfd5218e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac71 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.509081:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.509086:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074755.509304:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.509320:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.509330:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000001:00080000:0.0:1769074755.509386:0:6514:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589943785 00000004:00010000:0.0:1769074755.509409:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000004d4889a3 reply state 0000000040813015 transno 8589943786 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.509734:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.509740:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.509743:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1769074755.509746:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1769074755.510032:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62496:x1855007616629376:12345-192.168.204.60@tcp:36:mirror_io.0 Request processed in 1064us (1619us total) trans 8589943786 rc 0/0 00000100:00100000:0.0:1769074755.510041:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13191 00000020:00080000:0.0:1769074755.510047:0:6514:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1769074755.525268:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.536449:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.541213:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616630144 00000100:00100000:0.0:1769074755.541227:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616630144 00000100:00100000:0.0:1769074755.545546:0:24030:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616630272 00000100:00100000:0.0:1769074755.545561:0:24030:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616630272 00002000:00010000:0.0:1769074755.545620:0:24030:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/703 (4805728->18446744073709551615) 00002000:00100000:0.0:1769074755.545626:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8000 with x1855007616630272 ext(4805728->18446744073709551615) 00002000:00010000:0.0:1769074755.545642:0:24030:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497a8000. 00000100:00100000:0.0:1769074755.545671:0:24030:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616630400 00000100:00100000:0.0:1769074755.545675:0:24030:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616630400 00002000:00010000:0.0:1769074755.545688:0:24030:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/706 (4194304->18446744073709551615) 00002000:00100000:0.0:1769074755.545691:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9880 with x1855007616630400 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1769074755.545697:0:24030:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a9880. 00000100:00100000:0.0:1769074755.545713:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8231 00000100:00100000:0.0:1769074755.545721:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616630272:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:0.0:1769074755.546914:0:24030:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4808704 -> 4805728 00002000:00010000:0.0:1769074755.546920:0:24030:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074754 -> 1769074755 00002000:00010000:0.0:1769074755.546922:0:24030:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074755.546923:0:24030:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074754 -> 1769074755 00002000:00010000:0.0:1769074755.546925:0:24030:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074755.548744:0:24030:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2409:x1855007616630272:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 3010us (3397us total) trans 21474841109 rc 0/0 00000100:00100000:0.0:1769074755.549264:0:24030:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8231 00002000:00010000:0.0:1769074755.549271:0:24030:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/703 (4805728->18446744073709551615) 00002000:00100000:0.0:1769074755.549275:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8000 with x1855007616630272 ext(4805728->18446744073709551615) 00002000:00010000:0.0:1769074755.549286:0:24030:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497a8000. 00000100:00100000:0.0:1769074755.553087:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616630528 00000100:00100000:0.0:1769074755.553098:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616630528 00000100:00100000:0.0:1769074755.553256:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13192 00000100:00100000:0.0:1769074755.553266:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62496:x1855007616630528:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074755.553292:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.553317:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52283 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879acd3 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.553414:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd5228a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.553543:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd5228a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.553556:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd5228a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074755.553588:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:0.0:1769074755.553593:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074755.553622:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd5228a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.553649:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52283 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acd3 expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.553668:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd5228a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acd3 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.553678:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.553681:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074755.553685:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c800, rc 0) 00000100:00100000:0.0:1769074755.554592:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62496:x1855007616630528:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 675us (1212us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.554616:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13192 00000100:00100000:0.0:1769074755.560396:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616630784 00000100:00100000:0.0:1769074755.560408:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616630784 00000100:00100000:0.0:1769074755.560492:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13193 00000100:00100000:0.0:1769074755.560502:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62496:x1855007616630784:12345-192.168.204.60@tcp:36:mirror_io.0 00010000:00010000:0.0:1769074755.560558:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074755.560583:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd5228a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acd3 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.560611:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.560617:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074755.560784:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.560799:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.560850:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000001:00080000:0.0:1769074755.560898:0:6514:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589943787 00000004:00010000:0.0:1769074755.560931:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000000d53929a reply state 00000000905e46da transno 8589943788 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.560953:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.560960:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.560963:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1769074755.560966:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1769074755.561121:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62496:x1855007616630784:12345-192.168.204.60@tcp:36:mirror_io.0 Request processed in 618us (1228us total) trans 8589943788 rc 0/0 00000100:00100000:0.0:1769074755.561129:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13193 00000020:00080000:0.0:1769074755.561134:0:6514:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1769074755.565151:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616630912 00000100:00100000:1.0:1769074755.565161:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616630912 00000100:00100000:1.0:1769074755.565252:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13529 00000100:00100000:1.0:1769074755.565261:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62496:x1855007616630912:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074755.565278:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074755.565298:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879acda expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074755.565321:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bd:0x0] initial LVB size: 4194304, mtime: 0x6971f042, atime: 0x0, ctime: 0x6971f042, blocks: 0x1fe8 00010000:00010000:1.0:1769074755.565340:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074755.565343:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074755.565352:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.565367:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.565379:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47e00, rc 0) 00000100:00100000:1.0:1769074755.565525:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62496:x1855007616630912:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 263us (801us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074755.565532:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13529 00000100:00100000:0.0:1769074755.568188:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616631040 00000100:00100000:0.0:1769074755.568196:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616631040 00000100:00100000:0.0:1769074755.568245:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13530 00000100:00100000:0.0:1769074755.568254:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62496:x1855007616631040:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074755.568270:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.568292:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ace1 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074755.568315:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c3:0x0] initial LVB size: 8388608, mtime: 0x6971f043, atime: 0x0, ctime: 0x6971f043, blocks: 0x2000 00010000:00010000:0.0:1769074755.568335:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.568337:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.568346:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.568361:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.568372:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910dc00, rc 0) 00000100:00100000:0.0:1769074755.568434:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62496:x1855007616631040:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 181us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.568441:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13530 00000100:00100000:0.0:1769074755.570980:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.575124:0:24030:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616631296 00000100:00100000:0.0:1769074755.575136:0:24030:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616631296 00002000:00010000:0.0:1769074755.575187:0:24030:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/708 (4805728->18446744073709551615) 00002000:00100000:0.0:1769074755.575193:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1c00 with x1855007616631296 ext(4805728->18446744073709551615) 00002000:00010000:0.0:1769074755.575209:0:24030:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c1c00. 00000100:00100000:0.0:1769074755.575244:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8233 00000100:00100000:0.0:1769074755.575254:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616631296:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:0.0:1769074755.575609:0:24030:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4808704 -> 4805728 00002000:00010000:0.0:1769074755.575614:0:24030:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074755.575616:0:24030:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074755.575619:0:24030:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074755.575621:0:24030:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074755.575698:0:24030:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616631296:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 444us (776us total) trans 17179874430 rc 0/0 00000100:00100000:0.0:1769074755.575706:0:24030:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8233 00002000:00010000:0.0:1769074755.575711:0:24030:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/708 (4805728->18446744073709551615) 00002000:00100000:0.0:1769074755.575714:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1c00 with x1855007616631296 ext(4805728->18446744073709551615) 00002000:00010000:0.0:1769074755.575719:0:24030:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c1c00. 00000100:00100000:1.0:1769074755.578798:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616631424 00000100:00100000:1.0:1769074755.578828:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616631424 00002000:00010000:1.0:1769074755.578867:0:16437:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/704 (4194304->18446744073709551615) 00002000:00100000:1.0:1769074755.578872:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1180 with x1855007616631424 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1769074755.578884:0:16437:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c1180. 00000100:00100000:1.0:1769074755.578917:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8234 00000100:00100000:1.0:1769074755.578925:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616631424:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:1.0:1769074755.579075:0:16437:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074755.579079:0:16437:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074746 -> 1769074755 00002000:00010000:1.0:1769074755.579081:0:16437:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.579084:0:16437:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074746 -> 1769074755 00002000:00010000:1.0:1769074755.579086:0:16437:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074755.579149:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616631424:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 223us (470us total) trans 21474841110 rc 0/0 00000100:00100000:1.0:1769074755.579157:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8234 00002000:00010000:1.0:1769074755.579161:0:16437:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/704 (4194304->18446744073709551615) 00002000:00100000:1.0:1769074755.579163:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1180 with x1855007616631424 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1769074755.579168:0:16437:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c1180. 00000100:00100000:0.0:1769074755.587157:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616631680 00000100:00100000:0.0:1769074755.587169:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616631680 00000100:00100000:0.0:1769074755.587235:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4588 00000100:00100000:0.0:1769074755.587245:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62496:x1855007616631680:12345-192.168.204.60@tcp:35:mirror_io.0 00000004:00010000:0.0:1769074755.587321:0:6515:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x200000405:0x1fb:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90400/0x7fe2323adfd5213a lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac5c expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.587383:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.587398:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd52180 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac6a expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.587407:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.587456:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd52180 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac6a expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.587472:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd52180 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac6a expref: 15 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.587492:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625036928:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.587602:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00010000:00010000:1.0:1769074755.594156:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd52180 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac6a expref: 15 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074755.594182:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625036928:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074755.594210:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074755.594220:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074755.594226:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000040:00080000:1.0:1769074755.594660:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2365, 88 off202328 00000004:00080000:1.0:1769074755.594671:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743436863141ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074755.594706:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2370, 88 off202504 00000004:00080000:1.0:1769074755.594712:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743436904765ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074755.594858:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2371, 88 off202592 00000004:00080000:1.0:1769074755.594865:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743437057653ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074755.595655:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2366, 88 off202416 00000004:00080000:1.0:1769074755.595664:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743437856501ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074755.595686:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2367, 88 off202504 00000004:00080000:1.0:1769074755.595691:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743437883826ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074755.595701:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2372, 88 off202680 00000004:00080000:1.0:1769074755.595705:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743437898118ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074755.595966:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2373, 88 off202768 00000004:00080000:1.0:1769074755.595973:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743438166324ns, next at 1706953247299ns, rc = 0 00000001:00080000:0.0:1769074755.601046:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943789 is committed 00000004:00080000:0.0:1769074755.601062:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1882 committed 00010000:00010000:1.0:1769074755.601102:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601115:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601123:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601128:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910ca00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074755.601133:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601141:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601165:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074755.601168:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074755.601173:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601179:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52252 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601201:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601206:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601211:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601217:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074755.601220:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601225:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601232:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074755.601234:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074755.601237:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.601242:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52291 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:1.0:1769074755.601960:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037056:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602048:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602080:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037568:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602099:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602119:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1769074755.602125:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037440:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602145:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602162:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037952:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602179:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602208:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037184:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602227:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602243:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf96d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037696:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602260:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602275:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1769074755.602280:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037312:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602298:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602315:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037824:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074755.602332:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.602366:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037056 00000100:00100000:1.0:1769074755.602374:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037056 00000100:00100000:1.0:1769074755.602448:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037568 00000100:00100000:1.0:1769074755.602452:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037568 00000100:00100000:1.0:1769074755.602470:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037440 00000100:00100000:1.0:1769074755.602472:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037440 00000100:00100000:1.0:1769074755.602485:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037952 00000100:00100000:1.0:1769074755.602488:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037952 00000100:00100000:1.0:1769074755.602497:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037184 00000100:00100000:1.0:1769074755.602498:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037184 00000100:00100000:1.0:1769074755.602507:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037696 00000100:00100000:1.0:1769074755.602510:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037696 00000100:00100000:1.0:1769074755.602517:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037312 00000100:00100000:1.0:1769074755.602519:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037312 00000100:00100000:1.0:1769074755.602526:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625037824 00000100:00100000:1.0:1769074755.602528:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625037824 00000100:00100000:1.0:1769074755.602545:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13532 00000100:00100000:1.0:1769074755.602552:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625037056:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.602733:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074755.602737:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.602740:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074755.602742:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.602744:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074755.602784:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625037056:12345-0@lo:2:osp-syn-0-0.0 Request processed in 231us (734us total) trans 21474841111 rc 0/0 00000100:00100000:1.0:1769074755.602791:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13532 00000100:00100000:1.0:1769074755.602839:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13533 00000100:00100000:1.0:1769074755.602846:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625037568:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.602897:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:1.0:1769074755.602900:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.602903:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.602905:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.602907:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074755.602932:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625037568:12345-0@lo:2:osp-syn-0-0.0 Request processed in 87us (831us total) trans 21474841112 rc 0/0 00000100:00100000:1.0:1769074755.602938:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13533 00000100:00100000:1.0:1769074755.602950:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13534 00000100:00100000:1.0:1769074755.602956:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625037440:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.603006:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074755.603009:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.603010:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.603012:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.603015:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:1.0:1769074755.603039:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625037440:12345-0@lo:2:osp-syn-0-0.0 Request processed in 83us (893us total) trans 21474841113 rc 0/0 00000100:00100000:1.0:1769074755.603044:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13534 00000100:00100000:1.0:1769074755.603055:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13535 00000100:00100000:1.0:1769074755.603061:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625037952:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.603110:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074755.603113:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.603115:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.603116:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.603118:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074755.603142:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625037952:12345-0@lo:2:osp-syn-0-0.0 Request processed in 81us (961us total) trans 21474841114 rc 0/0 00000100:00100000:1.0:1769074755.603148:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13535 00000100:00100000:1.0:1769074755.603160:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13536 00000100:00100000:1.0:1769074755.603165:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625037184:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.603211:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074755.603215:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.603217:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074755.603218:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.603221:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074755.603253:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625037184:12345-0@lo:2:osp-syn-0-0.0 Request processed in 88us (1024us total) trans 21474841115 rc 0/0 00000100:00100000:1.0:1769074755.603258:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13536 00000100:00100000:1.0:1769074755.603271:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13537 00000100:00100000:1.0:1769074755.603276:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625037696:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.603322:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074755.603325:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.603328:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.603330:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.603332:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074755.603353:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625037696:12345-0@lo:2:osp-syn-0-0.0 Request processed in 77us (1093us total) trans 21474841116 rc 0/0 00000100:00100000:1.0:1769074755.603358:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13537 00000100:00100000:1.0:1769074755.603368:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13538 00000100:00100000:1.0:1769074755.603373:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625037312:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.603411:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074755.603414:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.603416:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074751 -> 1769074751 00002000:00010000:1.0:1769074755.603419:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.603421:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:1.0:1769074755.603440:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625037312:12345-0@lo:2:osp-syn-0-0.0 Request processed in 67us (1141us total) trans 21474841117 rc 0/0 00000100:00100000:1.0:1769074755.603445:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13538 00000100:00100000:1.0:1769074755.603455:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13539 00000100:00100000:1.0:1769074755.603460:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625037824:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074755.603497:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074755.603499:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.603502:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.603504:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.603506:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074755.603525:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625037824:12345-0@lo:2:osp-syn-0-0.0 Request processed in 65us (1191us total) trans 21474841118 rc 0/0 00000100:00100000:1.0:1769074755.603530:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13539 00000100:00080000:1.0:1769074755.603564:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074755.603614:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841102 gen 2 00000004:00080000:1.0:1769074755.603619:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f95c00, transno 21474841095 00000004:00080000:1.0:1769074755.603628:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f96d80, transno 21474841096 00000004:00080000:1.0:1769074755.603630:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f94e00, transno 21474841097 00000004:00080000:1.0:1769074755.603633:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f96680, transno 21474841098 00000004:00080000:1.0:1769074755.603636:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f95880, transno 21474841099 00000004:00080000:1.0:1769074755.603638:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f95180, transno 21474841100 00000004:00080000:1.0:1769074755.603640:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f96a00, transno 21474841101 00000004:00080000:1.0:1769074755.603642:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f94380, transno 21474841102 00000100:00100000:1.0:1769074755.603648:0:3651:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b54cf95180 x1855007625037312/t21474841117(21474841117) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074771 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1769074755.603660:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf95180/2, rc 0, transno 21474841117 00000100:00100000:1.0:1769074755.603664:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037312:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603674:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf94700/2, rc 0, transno 21474841118 00000100:00100000:1.0:1769074755.603678:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037824:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603684:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5446c2d80/2, rc 0, transno 21474841115 00000100:00100000:1.0:1769074755.603687:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037184:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603694:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf96d80/2, rc 0, transno 21474841116 00000100:00100000:1.0:1769074755.603697:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf96d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625037696:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603716:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf96a00/2, rc 0, transno 21474841113 00000100:00100000:1.0:1769074755.603719:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037440:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603725:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf96300/2, rc 0, transno 21474841114 00000100:00100000:1.0:1769074755.603728:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037952:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603734:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5446c1500/2, rc 0, transno 21474841111 00000100:00100000:1.0:1769074755.603738:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037056:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074755.603744:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf96680/2, rc 0, transno 21474841112 00000100:00100000:1.0:1769074755.603747:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625037568:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074755.604658:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1877 committed 00000004:00080000:0.0:1769074755.604670:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1876 committed 00000004:00080000:0.0:1769074755.604673:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1875 committed 00000004:00080000:0.0:1769074755.604675:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1874 committed 00000004:00080000:0.0:1769074755.604677:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1881 committed 00000004:00080000:0.0:1769074755.604679:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1873 committed 00000004:00080000:0.0:1769074755.604681:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1880 committed 00000004:00080000:0.0:1769074755.604683:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1872 committed 00000004:00080000:0.0:1769074755.604685:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1879 committed 00000004:00080000:0.0:1769074755.604686:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1878 committed 00000004:00080000:0.0:1769074755.604689:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1877 committed 00000004:00080000:0.0:1769074755.604690:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1871 committed 00000004:00080000:0.0:1769074755.604692:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1870 committed 00000004:00080000:0.0:1769074755.604694:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1876 committed 00000004:00080000:0.0:1769074755.604696:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1875 committed 00000100:00100000:1.0:1769074755.604888:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038336:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074755.606420:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.606448:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038848:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074755.606471:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.606483:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1769074755.606488:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038208:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074755.606502:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.606511:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038720:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074755.606525:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.606545:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625038336 00000100:00100000:1.0:1769074755.606549:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625038336 00000100:00100000:1.0:1769074755.606571:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625038848 00000100:00100000:1.0:1769074755.606573:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625038848 00000100:00100000:1.0:1769074755.606583:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625038208 00000100:00100000:1.0:1769074755.606586:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625038208 00000100:00100000:1.0:1769074755.606604:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625038720 00000100:00100000:1.0:1769074755.606606:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625038720 00000100:00100000:1.0:1769074755.606617:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13544 00000100:00100000:1.0:1769074755.606622:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625038336:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074755.606684:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074755.606687:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.606690:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime from disk: 1769074751 -> 1769074751 00002000:00010000:1.0:1769074755.606692:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.606694:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb blocks from disk: 5296 -> 5296 00000100:00100000:1.0:1769074755.606724:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625038336:12345-0@lo:2:osp-syn-1-0.0 Request processed in 101us (305us total) trans 17179874435 rc 0/0 00000100:00100000:1.0:1769074755.606729:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13544 00000100:00100000:1.0:1769074755.606743:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13545 00000100:00100000:1.0:1769074755.606761:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625038848:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074755.606828:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074755.606831:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.606833:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.606835:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074755.606837:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:1.0:1769074755.606861:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625038848:12345-0@lo:2:osp-syn-1-0.0 Request processed in 113us (390us total) trans 17179874436 rc 0/0 00000100:00100000:1.0:1769074755.606867:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13545 00000100:00100000:1.0:1769074755.606880:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13546 00000100:00100000:1.0:1769074755.606886:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625038208:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074755.606931:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074755.606933:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074753 -> 1769074753 00002000:00010000:1.0:1769074755.606935:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074755.606937:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074753 -> 1769074753 00002000:00010000:1.0:1769074755.607154:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:1.0:1769074755.607183:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625038208:12345-0@lo:2:osp-syn-1-0.0 Request processed in 298us (680us total) trans 17179874437 rc 0/0 00000100:00100000:1.0:1769074755.607191:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13546 00000100:00100000:1.0:1769074755.607203:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13547 00000100:00100000:1.0:1769074755.607209:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625038720:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074755.607261:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074755.607264:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.607266:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.607268:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.607270:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074755.607293:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625038720:12345-0@lo:2:osp-syn-1-0.0 Request processed in 85us (767us total) trans 17179874438 rc 0/0 00000100:00100000:1.0:1769074755.607298:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13547 00000004:00080000:1.0:1769074755.607428:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 8 in progress, 4 in flight, 9 done 00000100:00100000:1.0:1769074755.607481:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074755.607491:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625039104:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074755.607519:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074755.607544:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625039104 00000100:00100000:1.0:1769074755.607549:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625039104 00000100:00100000:1.0:1769074755.607577:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13548 00000100:00100000:1.0:1769074755.607582:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625039104:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074755.607649:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074755.607654:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.607656:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074755.607658:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074755.607660:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074755.607695:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625039104:12345-0@lo:2:osp-syn-1-0.0 Request processed in 112us (175us total) trans 17179874439 rc 0/0 00000100:00100000:1.0:1769074755.607701:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13548 00000004:00080000:1.0:1769074755.607735:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475df800/2, rc 0, transno 17179874439 00000100:00100000:1.0:1769074755.607741:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625039104:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074755.608053:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef5500/2, rc 0, transno 17179874437 00000100:00100000:1.0:1769074755.608058:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038208:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074755.608066:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc000/2, rc 0, transno 17179874438 00000100:00100000:1.0:1769074755.608069:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038720:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074755.608078:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dd500/2, rc 0, transno 17179874435 00000100:00100000:1.0:1769074755.608081:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038336:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074755.608088:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dfb80/2, rc 0, transno 17179874436 00000100:00100000:1.0:1769074755.608091:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625038848:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074755.608419:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1869 committed 00000004:00010000:0.0:1769074755.608502:0:6515:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000025906f84 reply state 000000007fb6e457 transno 8589943789 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.608527:0:6515:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.608536:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.608539:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:0.0:1769074755.608542:0:6515:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1769074755.608549:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.608551:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074755.608557:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546f90400/0x7fe2323adfd5213a lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac5c expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00080000:0.0:1769074755.608667:0:6515:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:0.0:1769074755.608680:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62496:x1855007616631680:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 21437us (21713us total) trans 8589943789 rc 0/0 00000100:00100000:0.0:1769074755.608689:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4588 00010000:00010000:0.0:1769074755.608715:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.608722:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.608730:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.608736:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074755.608741:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.609056:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.609070:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.609073:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074755.609077:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074755.609083:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522ad lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00000100:00100000:1.0:1769074755.616925:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.621611:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616632192 00000100:00100000:1.0:1769074755.621623:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616632192 00000100:00100000:1.0:1769074755.621683:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4589 00000100:00100000:1.0:1769074755.621692:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62496:x1855007616632192:12345-192.168.204.60@tcp:35:mirror_io.0 00000100:00100000:1.0:1769074755.621981:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62496:x1855007616632192:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 289us (504us total) trans 8589943790 rc 0/0 00000100:00100000:1.0:1769074755.621992:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4589 00000100:00100000:1.0:1769074755.703371:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.703504:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616632704 00000100:00100000:0.0:1769074755.703517:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616632704 00000100:00100000:0.0:1769074755.703601:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13196 00000100:00100000:0.0:1769074755.703612:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62509:x1855007616632704:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074755.703634:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.703665:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522de lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879acfd expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.703758:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.703772:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.703784:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.703797:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.703803:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.703851:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704001:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704013:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704021:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074755.704035:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074755.704123:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704130:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704136:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704142:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704148:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074755.704153:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704159:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704175:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074755.704178:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074755.704185:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704191:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd522e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704206:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704211:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704219:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704223:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074755.704225:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704230:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704235:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47400) 00010000:00010000:0.0:1769074755.704238:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074755.704240:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704245:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd522ec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.704283:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c800/0x7fe2323adfd522de lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acfd expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.704294:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acfd expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.704301:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.704303:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074755.704306:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:0.0:1769074755.704382:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62509:x1855007616632704:12345-192.168.204.60@tcp:101:cat.0 Request processed in 771us (1021us total) trans 8589943792 rc 0/0 00000100:00100000:0.0:1769074755.704389:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13196 00000100:00100000:0.0:1769074755.716617:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616632832 00000100:00100000:0.0:1769074755.716629:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616632832 00000100:00100000:0.0:1769074755.716690:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13549 00000100:00100000:0.0:1769074755.716699:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:62507:x1855007616632832:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074755.716718:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.716738:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad04 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.716783:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52259 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aca9 expref: 14 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.716792:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074755.716795:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074755.716800:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.716883:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52259 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aca9 expref: 14 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.716898:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52259 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aca9 expref: 14 pid: 36516 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.716911:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625039232:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.716987:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074755.722178:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616633088 00000100:00100000:0.0:1769074755.722192:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616633088 00000100:00100000:0.0:1769074755.722271:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13197 00000100:00100000:0.0:1769074755.722281:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62505:x1855007616633088:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074755.722312:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.722338:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd52308 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad0b expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.722488:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074755.722504:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd522d0 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acf6 expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.722513:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acfd expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.722522:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 5/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074755.722560:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acfd expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.722571:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acfd expref: 8 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.722584:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625039360:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.722702:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd522d0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acf6 expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.722711:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd522d0 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acf6 expref: 223 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.722724:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625039488:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.723074:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b474840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074755.730124:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616633344 00000100:00100000:0.0:1769074755.730141:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616633344 00000100:00100000:0.0:1769074755.730230:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13198 00000100:00100000:0.0:1769074755.730241:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62503:x1855007616633344:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074755.730266:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.730289:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d600/0x7fe2323adfd52316 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad12 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.730408:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.730426:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.730434:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:1.0:1769074755.731161:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625039232:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074755.731197:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.731215:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c800, rc 0) 00000100:00100000:1.0:1769074755.731289:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:62507:x1855007616632832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 14590us (14932us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074755.731298:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13549 00010000:00010000:0.0:1769074755.742557:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd522f3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acfd expref: 10 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.742571:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625039360:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.742584:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd522d0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acf6 expref: 223 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.742604:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625039488:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.742625:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074755.742635:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074755.742639:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074755.742658:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074755.742846:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c200/0x7fe2323adfd52308 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.742861:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.742869:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.742871:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074755.742874:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:0.0:1769074755.742912:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62505:x1855007616633088:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20632us (21684us total) trans 8589943793 rc 0/0 00000100:00100000:0.0:1769074755.742919:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13197 00010000:00010000:0.0:1769074755.742948:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.742955:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074755.742966:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074755.743019:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d600/0x7fe2323adfd52316 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad12 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.743035:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad12 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.743042:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.743045:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074755.743047:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:0.0:1769074755.743070:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62503:x1855007616633344:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12831us (15076us total) trans 8589943794 rc 0/0 00000100:00100000:0.0:1769074755.743076:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13198 00000100:00100000:1.0:1769074755.743226:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616633600 00000100:00100000:1.0:1769074755.743232:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616633600 00000100:00100000:1.0:1769074755.743277:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13550 00000100:00100000:1.0:1769074755.743284:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+5:62509:x1855007616633600:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074755.743298:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074755.743312:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad19 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.743329:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074755.743331:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074755.743338:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad19 expref: 6 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.743349:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad19 expref: 6 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.743364:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47400, rc 0) 00000100:00100000:1.0:1769074755.743400:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+6:62509:x1855007616633600:12345-192.168.204.60@tcp:101:cat.0 Request processed in 117us (2120us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074755.743406:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13550 00000100:00100000:1.0:1769074755.755148:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616633984 00000100:00100000:1.0:1769074755.755159:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616633984 00000100:00100000:1.0:1769074755.755220:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13199 00000100:00100000:1.0:1769074755.755230:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62507:x1855007616633984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074755.755258:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074755.755281:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46600/0x7fe2323adfd52339 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad20 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.755339:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46600/0x7fe2323adfd52339 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad20 expref: 224 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.755354:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46600/0x7fe2323adfd52339 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad20 expref: 224 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.755375:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074755.755379:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074755.755382:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46600, rc 0) 00000100:00100000:1.0:1769074755.755455:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:62507:x1855007616633984:12345-192.168.204.60@tcp:101:cat.0 Request processed in 226us (414us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074755.755463:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13199 00000100:00100000:0.0:1769074755.755962:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf95880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625039616:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074755.755982:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074755.756021:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf97b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616633472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 14474us (14794us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.756030:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6687 00000100:00100000:0.0:1769074755.784163:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616634240 00000100:00100000:0.0:1769074755.784174:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616634240 00000100:00100000:0.0:1769074755.784231:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13551 00000100:00100000:0.0:1769074755.784240:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b359850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62507:x1855007616634240:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074755.784255:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.784275:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad2e expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.784305:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.784321:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074755.784324:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074755.784329:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.784370:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.784386:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 15 pid: 28955 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.784401:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625039744:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.784506:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074755.807478:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625039744:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.807518:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.807538:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d600, rc 0) 00000100:00100000:0.0:1769074755.807616:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b359850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62507:x1855007616634240:12345-192.168.204.60@tcp:101:cat.0 Request processed in 23375us (24643us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.807625:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13551 00000100:00100000:0.0:1769074755.817871:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616634368 00000100:00100000:0.0:1769074755.817886:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616634368 00010000:00010000:0.0:1769074755.817950:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 16 pid: 28955 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.817998:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6690 00000100:00100000:0.0:1769074755.818007:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2406:x1855007616634368:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074755.818026:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074755.818048:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 16 pid: 28955 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074755.818060:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 16 pid: 28955 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074755.818083:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52298 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acda expref: 16 pid: 28955 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074755.818099:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.818101:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.818109:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.818119:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.818166:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074755.818181:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625039872:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074755.818260:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074755.824613:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616634496 00000100:00100000:0.0:1769074755.824625:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616634496 00000100:00100000:0.0:1769074755.824687:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13552 00000100:00100000:0.0:1769074755.824700:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+5:62509:x1855007616634496:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074755.824723:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.824746:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad35 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.824773:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.824775:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074755.824783:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 6 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.824797:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 6 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.824844:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c200, rc 0) 00000100:00100000:0.0:1769074755.824918:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+6:62509:x1855007616634496:12345-192.168.204.60@tcp:101:cat.0 Request processed in 218us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.824926:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13552 00000100:00100000:0.0:1769074755.826280:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625039872:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074755.826306:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074755.826370:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616634368:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8363us (8702us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.826381:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6690 00000100:00100000:0.0:1769074755.871199:0:24030:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616634624 00000100:00100000:0.0:1769074755.871209:0:24030:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616634624 00002000:00100000:0.0:1769074755.871260:0:24030:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x2b9:0x0] (0->2097151) req@ffff89b54cf97100 x1855007616634624/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:291/0 lens 488/0 e 0 to 0 dl 1769074766 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074755.871272:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf97100 with x1855007616634624 ext(0->2097151) 00002000:00010000:0.0:1769074755.871278:0:24030:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf97100 00000100:00100000:0.0:1769074755.871309:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8235 00000100:00100000:0.0:1769074755.871317:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+6:2408:x1855007616634624:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074755.898615:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616634752 00000100:00100000:1.0:1769074755.898629:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616634752 00002000:00100000:1.0:1769074755.898686:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bd:0x0] (0->4194303) req@ffff89b54cf95c00 x1855007616634752/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:291/0 lens 488/0 e 0 to 0 dl 1769074766 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074755.898697:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf95c00 with x1855007616634752 ext(0->4194303) 00002000:00010000:1.0:1769074755.898704:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf95c00 00000100:00100000:1.0:1769074755.898737:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8236 00000100:00100000:1.0:1769074755.898747:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616634752:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074755.913771:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.917466:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616634880 00000100:00100000:0.0:1769074755.917483:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616634880 00000100:00100000:0.0:1769074755.917553:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13201 00000100:00100000:0.0:1769074755.917563:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62510:x1855007616634880:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074755.917599:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.917625:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd52355 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad3c expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.917695:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917711:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917723:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917739:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917746:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917753:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917862:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917876:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.917884:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad12 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.917892:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 5/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.917928:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad12 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.917940:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5231d lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad12 expref: 10 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.917953:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625040000:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.918058:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.918067:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 223 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.918076:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625040128:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.918100:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074755.923867:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.923897:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.924453:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625040000:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.924470:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625040128:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.924495:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074755.933842:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6691 00000100:00100000:0.0:1769074755.933877:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2408:x1855007616635136:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074755.933905:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074755.933917:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 224 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074755.933935:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 224 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074755.933960:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd5230f lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad0b expref: 224 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074755.933982:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.933988:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.933998:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.934003:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.934009:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074755.934504:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934514:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074755.934580:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:1.0:1769074755.934585:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074755.934642:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934655:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934667:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934673:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934679:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934691:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934709:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b08e800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074755.934714:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934724:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.934733:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074755.934736:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074755.935061:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935071:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd5235c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935097:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935103:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935108:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935114:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b679924a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074755.935116:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935122:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935134:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47400) 00010000:00010000:1.0:1769074755.935136:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074755.935139:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935143:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd52363 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074755.935200:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd52355 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad3c expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.935216:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad3c expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074755.935224:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074755.935226:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074755.935229:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924600, rc 0) 00000100:00100000:1.0:1769074755.935280:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62510:x1855007616634880:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 17717us (21518us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074755.935289:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13201 00000100:00100000:0.0:1769074755.936848:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2408:x1855007616635136:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 2975us (12978us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.936868:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6691 00000100:00100000:1.0:1769074755.968603:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616634752:12345-192.168.204.60@tcp:3:cat.0 Request processed in 69837us (72182us total) trans 0 rc 4194304/4194304 00000100:00100000:1.0:1769074755.968619:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8236 00002000:00100000:1.0:1769074755.968634:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bd:0x0] (0->4194303) req@ffff89b54cf95c00 x1855007616634752/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:291/0 lens 488/440 e 0 to 0 dl 1769074766 ref 1 fl Complete:/600/0 rc 4194304/4194304 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074755.968645:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf95c00 with x1855007616634752 ext(0->4194303) 00002000:00010000:1.0:1769074755.968651:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf95c00 00000100:00100000:1.0:1769074755.977781:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.978072:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616635264 00000100:00100000:0.0:1769074755.978084:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616635264 00000100:00100000:0.0:1769074755.978152:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13553 00000100:00100000:0.0:1769074755.978162:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+6:62509:x1855007616635264:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074755.978182:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.978202:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad43 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.978233:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52260 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb0 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.978241:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074755.978244:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074755.978247:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 7 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.978281:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52260 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb0 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.978294:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52260 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb0 expref: 13 pid: 36516 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.978306:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625040256:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.978397:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074755.981372:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074755.981629:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616635520 00000100:00100000:0.0:1769074755.981640:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616635520 00000100:00100000:0.0:1769074755.981702:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13202 00000100:00100000:0.0:1769074755.981713:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62510:x1855007616635520:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074755.981737:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074755.981765:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52378 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad4a expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.981971:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.981991:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad3c expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.982002:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.982039:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad3c expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.982050:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad3c expref: 11 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.982063:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625040384:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074755.982137:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074755.984570:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625040256:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.984612:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 7 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.984630:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3000, rc 0) 00000100:00100000:0.0:1769074755.984682:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62509:x1855007616635264:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6521us (6915us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074755.984690:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13553 00000100:00100000:1.0:1769074755.986218:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074755.990184:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:0.0:1769074755.990892:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd5236a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad3c expref: 11 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074755.990910:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625040384:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074755.990943:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.990953:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074755.990958:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074755.990979:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074755.991121:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52378 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad4a expref: 12 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.991135:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad4a expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074755.991142:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074755.991145:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074755.991148:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3c00, rc 0) 00000100:00100000:0.0:1769074755.991183:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62510:x1855007616635520:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 9472us (9824us total) trans 8589943795 rc 0/0 00000100:00100000:0.0:1769074755.991190:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13202 00000100:00100000:0.0:1769074756.006110:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616636032 00000100:00100000:0.0:1769074756.006121:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616636032 00000100:00100000:0.0:1769074756.006183:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13203 00000100:00100000:0.0:1769074756.006192:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62510:x1855007616636032:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074756.006216:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.006219:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1769074756.006245:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5237f lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad4a expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.006264:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074756.006269:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074756.006283:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd5238d lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad51 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.006390:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006405:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006415:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006429:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006436:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006442:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006451:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b64eff2800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074756.006454:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006461:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006469:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074756.006472:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074756.006475:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006480:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd52394 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.006506:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd5238d lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad51 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.006513:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074756.006516:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074756.006519:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd5238d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad51 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.006526:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3400, rc 0) 00000100:00100000:0.0:1769074756.006602:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62510:x1855007616636032:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 411us (676us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074756.006609:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13203 00000100:00100000:1.0:1769074756.017879:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.022428:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:3.0:1769074756.022779:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.022787:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.022793:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 5/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.022874:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.022888:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 9 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:3.0:1769074756.022902:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf95500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625040640:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.023010:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.023018:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 10 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:3.0:1769074756.023026:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf97b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625040768:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074756.023062:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074756.030197:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf95500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625040640:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074756.030214:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf97b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625040768:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.030241:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074756.045349:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.048599:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616636160 00000100:00100000:1.0:1769074756.048612:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616636160 00002000:00100000:1.0:1769074756.048680:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bd:0x0] (1048576->2097151) req@ffff89b6491edf80 x1855007616636160/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.048692:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491edf80 with x1855007616636160 ext(1048576->2097151) 00002000:00010000:1.0:1769074756.048698:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491edf80 00000100:00100000:1.0:1769074756.048738:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8237 00000100:00100000:1.0:1769074756.048748:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2406:x1855007616636160:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.052904:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637056 00000100:00100000:0.0:1769074756.052918:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637056 00000100:00100000:1.0:1769074756.053955:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13205 00000100:00100000:1.0:1769074756.053967:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:62507:x1855007616637056:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.053990:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.054013:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad5f expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.054068:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.054080:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 4/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 225 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.054106:0:6514:0:(ldlm_request.c:362:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.054119:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 3/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 225 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.054132:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.054134:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074756.054137:0:6514:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.054143:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164800, rc 0) 00000100:00100000:1.0:1769074756.054212:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:62507:x1855007616637056:12345-192.168.204.60@tcp:101:cat.0 Request processed in 246us (8874us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.054219:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13205 00000100:00100000:3.0:1769074756.059773:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616636928 00000100:00100000:3.0:1769074756.059787:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616636928 00010000:00010000:3.0:1769074756.059863:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 11 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:3.0:1769074756.059904:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616636800 00000100:00100000:3.0:1769074756.059908:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616636800 00010000:00010000:3.0:1769074756.059922:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 10 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:3.0:1769074756.059963:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6695 00000100:00100000:3.0:1769074756.059974:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616636928:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.059990:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074756.059998:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 11 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.060010:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 11 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.060035:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd522b4 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acef expref: 11 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.060052:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074756.060057:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.060066:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.060075:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074756.060127:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616636928:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 155us (11640us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.060134:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6695 00000100:00100000:3.0:1769074756.060153:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6696 00000100:00100000:3.0:1769074756.060160:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616636800:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.060168:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074756.060172:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 10 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.060183:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 10 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.060198:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52340 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad27 expref: 10 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.060207:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074756.060211:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.060219:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40290400000020 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.060227:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40290400000020 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.060236:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 224 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.060262:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074756.060283:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616636800:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 123us (8626us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.060289:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6696 00010000:00010000:3.0:1769074756.060311:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074756.060317:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 6512 initiator: MDT0 00000040:00080000:3.0:1769074756.060619:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2374, 88 off203120 00000004:00080000:3.0:1769074756.060629:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743902821795ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.060691:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2378, 88 off203208 00000004:00080000:3.0:1769074756.060698:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743902890790ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060732:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2379, 88 off203296 00000004:00080000:3.0:1769074756.060736:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743902929131ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060755:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2375, 88 off203208 00000004:00080000:3.0:1769074756.060759:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743902952225ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.060776:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2376, 88 off203296 00000004:00080000:3.0:1769074756.060780:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743902973040ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.060796:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2380, 88 off203384 00000004:00080000:3.0:1769074756.060800:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743902993134ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060874:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2381, 88 off203472 00000004:00080000:3.0:1769074756.060879:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743903072422ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060897:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2382, 88 off203560 00000004:00080000:3.0:1769074756.060901:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743903094359ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060916:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2377, 88 off203384 00000004:00080000:3.0:1769074756.060921:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743903113922ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.060936:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2383, 88 off203648 00000004:00080000:3.0:1769074756.060940:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743903133272ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060956:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2378, 88 off203472 00000004:00080000:3.0:1769074756.060960:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743903153562ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.060978:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2384, 88 off203736 00000004:00080000:3.0:1769074756.060982:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743903175139ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074756.060998:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2379, 88 off203560 00000004:00080000:3.0:1769074756.061002:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743903195565ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.061018:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2380, 88 off203648 00000004:00080000:3.0:1769074756.061022:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743903215152ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.061038:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2381, 88 off203736 00000004:00080000:3.0:1769074756.061042:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743903235390ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.061058:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2382, 88 off203824 00000004:00080000:3.0:1769074756.061062:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1743903255402ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074756.061080:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2385, 88 off203824 00000004:00080000:3.0:1769074756.061085:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1743903277904ns, next at 1706953247299ns, rc = 0 00000100:00100000:1.0:1769074756.067253:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637184 00000100:00100000:1.0:1769074756.067268:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637184 00000100:00100000:1.0:1769074756.067327:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13206 00000100:00100000:1.0:1769074756.067337:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62503:x1855007616637184:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074756.067356:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.067376:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd523b0 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad66 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.067441:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd523b0 lrc: 3/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad66 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.067453:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd523b0 lrc: 3/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad66 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.067467:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.067470:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074756.067474:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 225 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.067482:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164000, rc 0) 00000100:00100000:1.0:1769074756.067871:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62503:x1855007616637184:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 534us (8474us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.067879:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13206 00000100:00100000:1.0:1769074756.069512:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625040896:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074756.069541:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069565:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041408:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074756.069581:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069610:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041920:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074756.069628:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069642:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625042560:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069662:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069678:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625043072:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069694:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069713:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:1.0:1769074756.069718:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041024:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074756.069736:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069752:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041536:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074756.069773:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069789:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfefb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625042176:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069844:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069861:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625042688:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069879:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069910:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042304:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069932:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069943:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042816:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069960:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.069972:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1769074756.069977:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042432:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.069994:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.070005:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042944:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.070018:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.070038:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625040896 00000100:00100000:1.0:1769074756.070043:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625040896 00000100:00100000:1.0:1769074756.070070:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625041408 00000100:00100000:1.0:1769074756.070073:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625041408 00000100:00100000:1.0:1769074756.070082:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625041920 00000100:00100000:1.0:1769074756.070086:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625041920 00000100:00100000:1.0:1769074756.070094:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042560 00000100:00100000:1.0:1769074756.070096:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042560 00000100:00100000:1.0:1769074756.070104:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625043072 00000100:00100000:1.0:1769074756.070106:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625043072 00000100:00100000:1.0:1769074756.070115:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625041024 00000100:00100000:1.0:1769074756.070117:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625041024 00000100:00100000:1.0:1769074756.070126:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625041536 00000100:00100000:1.0:1769074756.070128:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625041536 00000100:00100000:1.0:1769074756.070137:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042176 00000100:00100000:1.0:1769074756.070139:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042176 00000100:00100000:1.0:1769074756.070146:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042688 00000100:00100000:1.0:1769074756.070148:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042688 00000100:00100000:1.0:1769074756.070155:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042304 00000100:00100000:1.0:1769074756.070157:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042304 00000100:00100000:1.0:1769074756.070165:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042816 00000100:00100000:1.0:1769074756.070167:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042816 00000100:00100000:1.0:1769074756.070174:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042432 00000100:00100000:1.0:1769074756.070176:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042432 00000100:00100000:1.0:1769074756.070184:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625042944 00000100:00100000:1.0:1769074756.070185:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625042944 00000100:00100000:1.0:1769074756.070196:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13558 00000100:00100000:1.0:1769074756.070202:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+22:3651:x1855007625040896:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074756.070305:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:1.0:1769074756.070308:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.070310:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074756.070312:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.070314:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb blocks from disk: 2024 -> 2024 00000100:00100000:1.0:1769074756.070353:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+23:3651:x1855007625040896:12345-0@lo:2:osp-syn-1-0.0 Request processed in 151us (811us total) trans 17179874444 rc 0/0 00000100:00100000:1.0:1769074756.070361:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13558 00000100:00100000:1.0:1769074756.070379:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13559 00000100:00100000:1.0:1769074756.070384:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+22:3651:x1855007625041408:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074756.070434:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.070437:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.070439:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.070441:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.070443:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.070469:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+23:3651:x1855007625041408:12345-0@lo:2:osp-syn-1-0.0 Request processed in 84us (886us total) trans 17179874445 rc 0/0 00000100:00100000:1.0:1769074756.070474:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13559 00000100:00100000:1.0:1769074756.070487:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13560 00000100:00100000:1.0:1769074756.070493:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+22:3651:x1855007625041920:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074756.070541:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074756.070544:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.070545:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.070547:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.070549:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074756.070574:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+23:3651:x1855007625041920:12345-0@lo:2:osp-syn-1-0.0 Request processed in 81us (945us total) trans 17179874446 rc 0/0 00000100:00100000:1.0:1769074756.070579:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13560 00000100:00100000:1.0:1769074756.070605:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13561 00000100:00100000:1.0:1769074756.070611:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3651:x1855007625042560:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.070688:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.070691:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.070693:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.070695:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.070697:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:1.0:1769074756.070722:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625042560:12345-0@lo:2:osp-syn-0-0.0 Request processed in 111us (1058us total) trans 21474841119 rc 0/0 00000100:00100000:1.0:1769074756.070728:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13561 00000100:00100000:1.0:1769074756.070741:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13562 00000100:00100000:1.0:1769074756.070747:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3651:x1855007625043072:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.070802:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.070830:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.070833:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.070834:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.070837:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074756.070862:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625043072:12345-0@lo:2:osp-syn-0-0.0 Request processed in 116us (1167us total) trans 21474841120 rc 0/0 00000100:00100000:1.0:1769074756.070868:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13562 00000100:00100000:1.0:1769074756.070881:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13563 00000100:00100000:1.0:1769074756.070887:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+22:3651:x1855007625041024:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074756.070934:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.070937:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074753 -> 1769074753 00002000:00010000:1.0:1769074756.070939:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.070941:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074753 -> 1769074753 00002000:00010000:1.0:1769074756.070943:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:1.0:1769074756.070970:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+23:3651:x1855007625041024:12345-0@lo:2:osp-syn-1-0.0 Request processed in 83us (1232us total) trans 17179874447 rc 0/0 00000100:00100000:1.0:1769074756.070975:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13563 00000100:00100000:1.0:1769074756.070986:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13564 00000100:00100000:1.0:1769074756.070992:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+22:3651:x1855007625041536:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074756.071040:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.071043:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.071045:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.071048:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.071050:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074756.071076:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+23:3651:x1855007625041536:12345-0@lo:2:osp-syn-1-0.0 Request processed in 85us (1302us total) trans 17179874448 rc 0/0 00000100:00100000:1.0:1769074756.071082:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13564 00000100:00100000:1.0:1769074756.071095:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13565 00000100:00100000:1.0:1769074756.071101:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3651:x1855007625042176:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.071148:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.071150:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.071152:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074756.071154:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.071157:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.071183:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625042176:12345-0@lo:2:osp-syn-0-0.0 Request processed in 82us (1338us total) trans 21474841121 rc 0/0 00000100:00100000:1.0:1769074756.071189:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13565 00000100:00100000:1.0:1769074756.071202:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13566 00000100:00100000:1.0:1769074756.071208:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3651:x1855007625042688:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.071254:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:1.0:1769074756.071257:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.071258:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.071260:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.071262:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074756.071287:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625042688:12345-0@lo:2:osp-syn-0-0.0 Request processed in 80us (1407us total) trans 21474841122 rc 0/0 00000100:00100000:1.0:1769074756.071293:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13566 00000100:00100000:1.0:1769074756.071305:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13567 00000100:00100000:1.0:1769074756.071312:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3649:x1855007625042304:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.071363:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.071366:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.071368:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074756.071370:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.071372:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.071410:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625042304:12345-0@lo:2:osp-syn-0-0.0 Request processed in 99us (1476us total) trans 21474841123 rc 0/0 00000100:00100000:1.0:1769074756.071416:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13567 00000100:00100000:1.0:1769074756.071428:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13568 00000100:00100000:1.0:1769074756.071433:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3649:x1855007625042816:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.071481:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.071483:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.071485:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.071487:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.071489:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.071514:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625042816:12345-0@lo:2:osp-syn-0-0.0 Request processed in 81us (1552us total) trans 21474841124 rc 0/0 00000100:00100000:1.0:1769074756.071519:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13568 00000100:00100000:1.0:1769074756.071530:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13569 00000100:00100000:1.0:1769074756.071536:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3649:x1855007625042432:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.071580:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.071584:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.071599:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074751 -> 1769074751 00002000:00010000:1.0:1769074756.071602:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.071605:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:1.0:1769074756.071630:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625042432:12345-0@lo:2:osp-syn-0-0.0 Request processed in 94us (1635us total) trans 21474841125 rc 0/0 00000100:00100000:1.0:1769074756.071636:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13569 00000100:00100000:1.0:1769074756.071649:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13570 00000100:00100000:1.0:1769074756.071654:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+20:3649:x1855007625042944:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.071701:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074756.071705:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.071707:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.071709:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.071711:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074756.071738:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625042944:12345-0@lo:2:osp-syn-0-0.0 Request processed in 84us (1718us total) trans 21474841126 rc 0/0 00000100:00100000:1.0:1769074756.071745:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13570 00000004:00080000:1.0:1769074756.071868:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cfee300/2, rc 0, transno 21474841125 00000100:00100000:1.0:1769074756.071874:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042432:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.071884:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54ce9d500/2, rc 0, transno 21474841126 00000100:00100000:1.0:1769074756.071888:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042944:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.071896:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cfef100/2, rc 0, transno 21474841123 00000100:00100000:1.0:1769074756.071899:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042304:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.071907:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54ce9d180/2, rc 0, transno 21474841124 00000100:00100000:1.0:1769074756.071910:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625042816:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.071942:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491eea00/2, rc 0, transno 17179874447 00000100:00100000:1.0:1769074756.071947:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041024:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074756.071956:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54510aa00/2, rc 0, transno 17179874448 00000100:00100000:1.0:1769074756.071960:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041536:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074756.071967:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cfefb80/2, rc 0, transno 21474841121 00000100:00100000:1.0:1769074756.071971:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfefb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625042176:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.071979:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54ce9f480/2, rc 0, transno 21474841122 00000100:00100000:1.0:1769074756.071982:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625042688:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.071990:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ece00/2, rc 0, transno 17179874444 00000100:00100000:1.0:1769074756.071992:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625040896:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074756.072000:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54510ad80/2, rc 0, transno 17179874445 00000100:00100000:1.0:1769074756.072002:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041408:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074756.072010:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d4700/2, rc 0, transno 17179874446 00000100:00100000:1.0:1769074756.072013:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625041920:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074756.072021:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54ce9e680/2, rc 0, transno 21474841119 00000100:00100000:1.0:1769074756.072023:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625042560:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.072030:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54538ad80/2, rc 0, transno 21474841120 00000100:00100000:1.0:1769074756.072033:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625043072:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.074970:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2406:x1855007616636160:12345-192.168.204.60@tcp:3:cat.0 Request processed in 26211us (26538us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074756.074983:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8237 00002000:00100000:1.0:1769074756.074995:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bd:0x0] (1048576->2097151) req@ffff89b6491edf80 x1855007616636160/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.075004:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491edf80 with x1855007616636160 ext(1048576->2097151) 00002000:00010000:1.0:1769074756.075010:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491edf80 00000100:00100000:0.0:1769074756.083625:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625042048:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074756.083639:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625043200:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074756.083686:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5239b lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 223 pid: 6512 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1769074756.083714:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 222 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.083747:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:1.0:1769074756.084133:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637312 00000100:00100000:1.0:1769074756.084144:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637312 00000100:00100000:1.0:1769074756.084199:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637440 00000100:00100000:1.0:1769074756.084202:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637440 00000100:00100000:1.0:1769074756.084221:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4591 00000100:00100000:1.0:1769074756.084229:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62503:x1855007616637312:12345-192.168.204.60@tcp:49:multiop.0 00000100:00100000:1.0:1769074756.089265:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62503:x1855007616637312:12345-192.168.204.60@tcp:49:multiop.0 Request processed in 5022us (5743us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.089279:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4591 00000100:00100000:1.0:1769074756.089320:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4592 00000100:00100000:1.0:1769074756.089341:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62507:x1855007616637440:12345-192.168.204.60@tcp:49:cat.0 00000100:00100000:1.0:1769074756.089485:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62507:x1855007616637440:12345-192.168.204.60@tcp:49:cat.0 Request processed in 156us (5938us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.089491:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4592 00000100:00100000:0.0:1769074756.092537:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62505:x1855007616636672:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 69897us (70116us total) trans 8589943797 rc 0/0 00000100:00100000:0.0:1769074756.092551:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13204 00000100:00100000:0.0:1769074756.092654:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.092873:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.093683:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637696 00000100:00100000:3.0:1769074756.093696:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637696 00000100:00100000:3.0:1769074756.093751:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4593 00000100:00100000:3.0:1769074756.093760:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62505:x1855007616637696:12345-192.168.204.60@tcp:49:multiop.0 00000100:00100000:3.0:1769074756.093931:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62505:x1855007616637696:12345-192.168.204.60@tcp:49:multiop.0 Request processed in 172us (1058us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.095541:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4593 00000100:00100000:1.0:1769074756.096262:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13571 00000100:00100000:1.0:1769074756.096272:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+6:62509:x1855007616637568:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.096289:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.096310:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad6d expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.096345:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 14 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.096353:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074756.096355:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074756.096358:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.096398:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 14 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.096426:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 14 pid: 8399 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.096439:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625043328:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.096529:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.101426:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637824 00000100:00100000:0.0:1769074756.101444:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637824 00010000:00010000:0.0:1769074756.101495:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 15 pid: 8399 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074756.101532:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6697 00000100:00100000:0.0:1769074756.101543:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616637824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.101560:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.101583:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 15 pid: 8399 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074756.101612:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 15 pid: 8399 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074756.101649:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.101652:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.101663:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.101676:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.101727:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.101748:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625043456:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074756.101880:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00010000:00010000:3.0:1769074756.103745:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd52267 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acb7 expref: 15 pid: 8399 timeout: 1843 lvb_type: 0 00000100:00100000:3.0:1769074756.103768:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625043328:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.103797:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.103958:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164200, rc 0) 00000100:00100000:0.0:1769074756.104230:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62509:x1855007616637568:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7958us (11574us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.104239:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13571 00000100:00100000:1.0:1769074756.104443:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625043456:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074756.104458:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.104846:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2409:x1855007616637824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3303us (4003us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.104855:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6697 00000100:00100000:0.0:1769074756.113870:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.113920:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.114025:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616638080 00000100:00100000:1.0:1769074756.114036:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616638080 00000100:00100000:1.0:1769074756.114093:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616637952 00000100:00100000:1.0:1769074756.114097:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616637952 00000100:00100000:1.0:1769074756.114128:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13572 00000100:00100000:1.0:1769074756.114137:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:62505:x1855007616638080:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074756.114157:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.114177:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad7b expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.114208:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac78 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.114222:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad19 expref: 6 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.114228:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.114230:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.114234:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [643072->1048575] (req 643072->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad7b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.114268:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad19 expref: 6 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.114281:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad19 expref: 6 pid: 27844 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.114295:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625043584:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.114385:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac78 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.114395:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac78 expref: 13 pid: 27844 timeout: 1843 lvb_type: 1 00000100:00100000:1.0:1769074756.114403:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625043712:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.114432:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.114498:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13573 00000100:00100000:0.0:1769074756.114506:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62507:x1855007616637952:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074756.114514:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.114523:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad74 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.114534:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.114536:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.114543:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.114555:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.114566:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3e00, rc 0) 00000100:00100000:0.0:1769074756.114608:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62507:x1855007616637952:12345-192.168.204.60@tcp:101:cat.0 Request processed in 102us (686us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.114615:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13573 00000100:00100000:1.0:1769074756.139977:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616638592 00000100:00100000:1.0:1769074756.139982:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616638592 00000100:00100000:1.0:1769074756.140020:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616638848 00000100:00100000:1.0:1769074756.140024:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616638848 00000100:00100000:1.0:1769074756.140048:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13207 00000100:00100000:1.0:1769074756.140055:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62509:x1855007616638592:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.140074:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.140094:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad82 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140156:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140171:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140189:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.140193:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074756.140196:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924a00, rc 0) 00000100:00100000:1.0:1769074756.140228:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62509:x1855007616638592:12345-192.168.204.60@tcp:101:cat.0 Request processed in 173us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.140235:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13207 00000100:00100000:1.0:1769074756.140253:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13208 00000100:00100000:1.0:1769074756.140260:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62503:x1855007616638848:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074756.140267:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.140269:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074756.140279:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd523b0 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad66 expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140295:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.140299:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074756.140306:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd523d3 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad89 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140333:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.140346:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 221 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140354:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 221 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140361:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140368:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 6/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.140391:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140403:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 9 pid: 6513 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.140415:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625043968:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.140469:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 221 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140479:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 221 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.140489:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625044096:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.140518:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 221 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140526:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 221 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.140535:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625044224:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.140556:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397600 going to sleep for 11 seconds 00010000:00010000:1.0:1769074756.140608:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd52332 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad19 expref: 6 pid: 27844 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.140619:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625043584:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.140631:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546f91800/0x7fe2323adfd52195 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac78 expref: 14 pid: 27844 timeout: 1843 lvb_type: 1 00000100:00100000:1.0:1769074756.140638:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625043712:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.140665:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 643072->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad7b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.140674:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924600, rc 0) 00000100:00100000:1.0:1769074756.140699:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62505:x1855007616638080:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 26564us (26892us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.140707:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13572 00000100:00100000:3.0:1769074756.145061:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616638208 00000100:00100000:3.0:1769074756.145077:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616638208 00002000:00100000:3.0:1769074756.145133:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54d101180 x1855007616638208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.145144:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101180 with x1855007616638208 ext(2097152->3145727) 00002000:00010000:3.0:1769074756.145150:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d101180 00000100:00100000:3.0:1769074756.145186:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8238 00000100:00100000:3.0:1769074756.145195:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616638208:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074756.154647:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616638976 00000100:00100000:1.0:1769074756.154659:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616638976 00000100:00100000:1.0:1769074756.154717:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13574 00000100:00100000:1.0:1769074756.154727:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+6:62509:x1855007616638976:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.154746:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.154766:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad90 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.154795:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.154798:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.154840:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.154858:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 7 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.154870:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164000, rc 0) 00000100:00100000:1.0:1769074756.154934:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62509:x1855007616638976:12345-192.168.204.60@tcp:101:cat.0 Request processed in 205us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.154940:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13574 00000100:00100000:0.0:1769074756.160037:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625043840:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074756.160068:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.160132:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616638464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 20676us (21111us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.160143:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6699 00000100:00100000:3.0:1769074756.172366:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.172410:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.172443:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.172487:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616639232 00000100:00100000:3.0:1769074756.172494:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616639232 00000100:00100000:0.0:1769074756.172501:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616639616 00000100:00100000:0.0:1769074756.172507:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616639616 00010000:00010000:3.0:1769074756.172536:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 223 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074756.172539:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 223 pid: 6514 timeout: 1843 lvb_type: 0 00000100:00100000:3.0:1769074756.172577:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6700 00000100:00100000:3.0:1769074756.172598:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2408:x1855007616639232:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.172615:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074756.172623:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 223 pid: 6512 timeout: 1843 lvb_type: 0 00000100:00100000:1.0:1769074756.172624:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616639360 00000100:00100000:1.0:1769074756.172631:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616639360 00010000:00010000:3.0:1769074756.172639:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd523a2 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad58 expref: 223 pid: 6512 timeout: 1843 lvb_type: 0 00010000:00010000:3.0:1769074756.172667:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00000100:00100000:1.0:1769074756.172668:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13209 00010000:00010000:3.0:1769074756.172676:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00000100:00100000:1.0:1769074756.172681:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf97b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:62512:x1855007616639360:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074756.172690:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074756.172698:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074756.172710:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.172730:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd523e8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad97 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074756.172753:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2408:x1855007616639232:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 167us (342us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.172762:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6700 00000100:00100000:3.0:1769074756.172781:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6701 00000100:00100000:3.0:1769074756.172788:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:2408:x1855007616639616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.172798:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.172802:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.172839:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 222 pid: 6514 timeout: 1843 lvb_type: 0 00010000:00010000:1.0:1769074756.172848:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.172853:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 222 pid: 6514 timeout: 1843 lvb_type: 0 00010000:00010000:1.0:1769074756.172863:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.172872:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd523a9 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad5f expref: 222 pid: 6514 timeout: 1843 lvb_type: 0 00010000:00010000:1.0:1769074756.172883:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.172892:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.172893:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.172900:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.172904:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.172908:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074756.172915:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074756.172936:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:2408:x1855007616639616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 150us (493us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.172942:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6701 00010000:00010000:1.0:1769074756.173470:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074756.173480:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.173485:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x40 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074756.173500:0:6514:0:(ldlm_request.c:362:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210400000020 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.173508:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x40 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074756.176374:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616639104 00000100:00100000:0.0:1769074756.176379:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616639104 00010000:00010000:0.0:1769074756.177896:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 10 pid: 6513 timeout: 1843 lvb_type: 0 00000100:00100000:0.0:1769074756.177949:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6702 00000100:00100000:0.0:1769074756.177960:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616639104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.177977:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.177985:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 10 pid: 6513 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074756.177998:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 10 pid: 6513 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074756.178021:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd523cc lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad82 expref: 10 pid: 6513 timeout: 1843 lvb_type: 0 00010000:00010000:0.0:1769074756.178037:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074756.178041:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x40 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.178050:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210400000020 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.178057:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40290400000020 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.178064:0:6499:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40290400000020 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.178071:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x40 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.178077:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x40 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074756.178103:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.178141:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616639104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 184us (4104us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.178151:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6702 00010000:00010000:0.0:1769074756.178286:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.178295:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 6513 initiator: MDT0 00000040:00080000:0.0:1769074756.178634:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2383, 88 off203912 00000004:00080000:0.0:1769074756.178643:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744020835524ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178675:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2386, 88 off203912 00000004:00080000:0.0:1769074756.178680:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744020873383ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178698:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2387, 88 off204000 00000004:00080000:0.0:1769074756.178702:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744020894876ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178711:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2384, 88 off204000 00000004:00080000:0.0:1769074756.178715:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744020907714ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178723:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2385, 88 off204088 00000004:00080000:0.0:1769074756.178726:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744020919389ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178735:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2388, 88 off204088 00000004:00080000:0.0:1769074756.178738:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744020931309ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178746:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2389, 88 off204176 00000004:00080000:0.0:1769074756.178750:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744020942799ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178759:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2390, 88 off204264 00000004:00080000:0.0:1769074756.178762:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744020955251ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178771:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2386, 88 off204176 00000004:00080000:0.0:1769074756.178774:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744020966869ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178781:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2391, 88 off204352 00000004:00080000:0.0:1769074756.178784:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744020977163ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178794:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2387, 88 off204264 00000004:00080000:0.0:1769074756.178797:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744020990081ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178840:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2392, 88 off204440 00000004:00080000:0.0:1769074756.178846:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744021038658ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074756.178856:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2388, 88 off204352 00000004:00080000:0.0:1769074756.178859:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744021052574ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178868:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2389, 88 off204440 00000004:00080000:0.0:1769074756.178871:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744021064429ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178880:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2390, 88 off204528 00000004:00080000:0.0:1769074756.178884:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744021076791ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178892:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2391, 88 off204616 00000004:00080000:0.0:1769074756.178896:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1744021089104ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074756.178905:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2393, 88 off204528 00000004:00080000:0.0:1769074756.178908:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1744021101283ns, next at 1706953247299ns, rc = 0 00000100:00100000:0.0:1769074756.179019:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616639744 00000100:00100000:0.0:1769074756.179027:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616639744 00000100:00100000:0.0:1769074756.179080:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13210 00000100:00100000:0.0:1769074756.179088:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62509:x1855007616639744:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074756.179108:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.179130:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ad9e expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.179194:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.179207:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.179225:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074756.179228:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074756.179231:0:8855:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.179238:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3400, rc 0) 00000100:00100000:0.0:1769074756.179279:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62509:x1855007616639744:12345-192.168.204.60@tcp:101:cat.0 Request processed in 191us (1965us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.179285:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13210 00000001:00080000:0.0:1769074756.193297:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943798 is committed 00000004:00080000:0.0:1769074756.193318:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1898 committed 00000004:00080000:0.0:1769074756.193341:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1895 committed 00000004:00080000:0.0:1769074756.193355:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1894 committed 00000004:00080000:0.0:1769074756.193358:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1893 committed 00000004:00080000:0.0:1769074756.193360:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1892 committed 00000004:00080000:0.0:1769074756.193362:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1897 committed 00000004:00080000:0.0:1769074756.193364:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1891 committed 00000004:00080000:0.0:1769074756.193366:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1896 committed 00000004:00080000:0.0:1769074756.193368:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1890 committed 00000004:00080000:0.0:1769074756.193370:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1895 committed 00000004:00080000:0.0:1769074756.193372:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1894 committed 00000004:00080000:0.0:1769074756.193374:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1893 committed 00000004:00080000:0.0:1769074756.193376:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1889 committed 00000004:00080000:0.0:1769074756.193378:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1888 committed 00000004:00080000:0.0:1769074756.193380:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1892 committed 00000004:00080000:0.0:1769074756.193382:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1891 committed 00000004:00080000:0.0:1769074756.193384:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1887 committed 00010000:00010000:3.0:1769074756.193522:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.193537:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074756.193541:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074756.193544:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x40 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193553:0:6513:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x53/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193558:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x53/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193565:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.193573:0:6513:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.193596:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.193640:0:6513:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074756.193656:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf95f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625044352:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074756.193760:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656049c0 going to sleep for 11 seconds 00010000:00010000:3.0:1769074756.193779:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x53/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193786:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x53/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074756.193870:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:3.0:1769074756.193876:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074756.193898:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x53/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193909:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193920:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193926:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193944:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193951:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193957:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b052e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074756.193961:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193968:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193981:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074756.193983:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074756.193987:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.193992:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd523ef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194008:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194013:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194018:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194023:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b052000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074756.194026:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194031:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194036:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6510d4600) 00010000:00010000:3.0:1769074756.194039:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074756.194042:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194048:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd523f6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074756.194086:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd523e8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad97 expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.194100:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad97 expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.194106:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074756.194108:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074756.194111:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:3.0:1769074756.194154:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf97b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62512:x1855007616639360:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 21474us (21798us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.194163:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13209 00000100:00100000:1.0:1769074756.197062:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625046016:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.197114:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.197140:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625046528:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.197160:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.197180:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1769074756.197185:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625045888:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.197207:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.197221:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625046400:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.197238:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.197266:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625046016 00000100:00100000:1.0:1769074756.197273:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625046016 00000100:00100000:1.0:1769074756.197307:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625046528 00000100:00100000:1.0:1769074756.197310:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625046528 00000100:00100000:1.0:1769074756.197323:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625045888 00000100:00100000:1.0:1769074756.197325:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625045888 00000100:00100000:1.0:1769074756.197340:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625046400 00000100:00100000:1.0:1769074756.197343:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625046400 00000100:00100000:1.0:1769074756.197359:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13584 00000100:00100000:1.0:1769074756.197365:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+24:3652:x1855007625046016:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.197477:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.197480:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.197482:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.197485:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.197486:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:1.0:1769074756.197520:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+25:3652:x1855007625046016:12345-0@lo:2:osp-syn-0-0.0 Request processed in 155us (405us total) trans 21474841127 rc 0/0 00000100:00100000:1.0:1769074756.197527:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13584 00000100:00100000:1.0:1769074756.197543:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13585 00000100:00100000:1.0:1769074756.197549:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+24:3652:x1855007625046528:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.197615:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.197618:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.197620:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.197622:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.197624:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074756.197650:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+25:3652:x1855007625046528:12345-0@lo:2:osp-syn-0-0.0 Request processed in 102us (489us total) trans 21474841128 rc 0/0 00000100:00100000:1.0:1769074756.197655:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13585 00000100:00100000:1.0:1769074756.197667:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13586 00000100:00100000:1.0:1769074756.197672:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+24:3652:x1855007625045888:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.197720:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.197723:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.197725:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074751 -> 1769074751 00002000:00010000:1.0:1769074756.197727:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.197729:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:1.0:1769074756.197752:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+25:3652:x1855007625045888:12345-0@lo:2:osp-syn-0-0.0 Request processed in 80us (544us total) trans 21474841129 rc 0/0 00000100:00100000:1.0:1769074756.197758:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13586 00000100:00100000:1.0:1769074756.197769:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13587 00000100:00100000:1.0:1769074756.197775:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+24:3652:x1855007625046400:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.197862:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074756.197865:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.197866:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.197868:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074756.197871:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074756.197894:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+25:3652:x1855007625046400:12345-0@lo:2:osp-syn-0-0.0 Request processed in 119us (655us total) trans 21474841130 rc 0/0 00000100:00100000:1.0:1769074756.197900:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13587 00000004:00080000:1.0:1769074756.197958:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc700/2, rc 0, transno 21474841129 00000100:00100000:1.0:1769074756.197964:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625045888:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.197974:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ec380/2, rc 0, transno 21474841130 00000100:00100000:1.0:1769074756.197977:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625046400:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.197986:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ded80/2, rc 0, transno 21474841127 00000100:00100000:1.0:1769074756.197989:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625046016:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.197995:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ef800/2, rc 0, transno 21474841128 00000100:00100000:1.0:1769074756.197997:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625046528:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.198015:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625045760:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.198050:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.198066:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625046272:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.198089:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.198105:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1769074756.198111:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625045632:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.198131:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.198144:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625046144:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074756.198162:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074756.198188:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625045760 00000100:00100000:1.0:1769074756.198194:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625045760 00000100:00100000:1.0:1769074756.198221:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625046272 00000100:00100000:1.0:1769074756.198224:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625046272 00000100:00100000:1.0:1769074756.198235:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625045632 00000100:00100000:1.0:1769074756.198238:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625045632 00000100:00100000:1.0:1769074756.198249:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625046144 00000100:00100000:1.0:1769074756.198252:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625046144 00000100:00100000:1.0:1769074756.198263:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13588 00000100:00100000:1.0:1769074756.198268:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b358450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+28:3650:x1855007625045760:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.198332:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.198335:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.198338:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074756.198341:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.198342:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.198371:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b358450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+29:3650:x1855007625045760:12345-0@lo:2:osp-syn-0-0.0 Request processed in 103us (322us total) trans 21474841131 rc 0/0 00000100:00100000:1.0:1769074756.198378:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13588 00000100:00100000:1.0:1769074756.198511:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13589 00000100:00100000:1.0:1769074756.198517:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+28:3650:x1855007625046272:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.198564:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074756.198567:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.198568:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.198570:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.198573:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.198612:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+29:3650:x1855007625046272:12345-0@lo:2:osp-syn-0-0.0 Request processed in 95us (522us total) trans 21474841132 rc 0/0 00000100:00100000:1.0:1769074756.198618:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13589 00000100:00100000:1.0:1769074756.198630:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13590 00000100:00100000:1.0:1769074756.198636:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+28:3650:x1855007625045632:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.198682:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074756.198685:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.198688:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:1.0:1769074756.198690:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074752 -> 1769074752 00002000:00010000:1.0:1769074756.198692:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074756.198716:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+29:3650:x1855007625045632:12345-0@lo:2:osp-syn-0-0.0 Request processed in 80us (583us total) trans 21474841133 rc 0/0 00000100:00100000:1.0:1769074756.198723:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13590 00000100:00100000:1.0:1769074756.198734:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13591 00000100:00100000:1.0:1769074756.198740:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+28:3650:x1855007625046144:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074756.198788:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:1.0:1769074756.198791:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.198794:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074756.198796:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074756.198798:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074756.198855:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+29:3650:x1855007625046144:12345-0@lo:2:osp-syn-0-0.0 Request processed in 115us (692us total) trans 21474841134 rc 0/0 00000100:00100000:1.0:1769074756.198861:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13591 00000004:00080000:1.0:1769074756.198908:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dd880/2, rc 0, transno 21474841133 00000100:00100000:1.0:1769074756.198914:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625045632:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.198922:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef6680/2, rc 0, transno 21474841134 00000100:00100000:1.0:1769074756.198925:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625046144:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.198931:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddc00/2, rc 0, transno 21474841131 00000100:00100000:1.0:1769074756.198934:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625045760:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074756.198942:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef5180/2, rc 0, transno 21474841132 00000100:00100000:1.0:1769074756.198944:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625046272:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074756.205669:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616639872 00000100:00100000:3.0:1769074756.205684:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616639872 00000100:00100000:3.0:1769074756.205770:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640000 00000100:00100000:3.0:1769074756.205774:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640000 00000100:00100000:3.0:1769074756.205830:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13592 00000100:00100000:3.0:1769074756.205842:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:62507:x1855007616639872:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074756.205863:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.205883:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ada5 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.205916:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.205919:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.205940:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.205959:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.205971:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030800, rc 0) 00000100:00100000:3.0:1769074756.206044:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:62507:x1855007616639872:12345-192.168.204.60@tcp:101:cat.0 Request processed in 203us (890us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.206053:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13592 00000100:00100000:3.0:1769074756.206075:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13593 00000100:00100000:3.0:1769074756.206081:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:62505:x1855007616640000:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074756.206089:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.206097:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adac expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.206116:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 12 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.206124:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.206126:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.206130:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->1310719] (req 1048576->1310719) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.206169:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 12 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.206185:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 12 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074756.206199:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625046656:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074756.206260:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.218040:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.218568:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640128 00000100:00100000:3.0:1769074756.218575:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640128 00000100:00100000:3.0:1769074756.218638:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4594 00000100:00100000:3.0:1769074756.218648:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62509:x1855007616640128:12345-192.168.204.60@tcp:49:cat.0 00000100:00100000:3.0:1769074756.218728:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62509:x1855007616640128:12345-192.168.204.60@tcp:49:cat.0 Request processed in 82us (700us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.218735:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4594 00000100:00100000:3.0:1769074756.222668:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.222743:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.222785:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.222854:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640256 00000100:00100000:3.0:1769074756.222864:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640256 00000100:00100000:3.0:1769074756.222920:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640512 00000100:00100000:3.0:1769074756.222923:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640512 00000100:00100000:3.0:1769074756.222955:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13211 00000100:00100000:3.0:1769074756.222965:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+220:62507:x1855007616640256:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074756.222985:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.223010:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adb3 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.223080:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 221 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.223100:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 221 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.223123:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074756.223126:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074756.223130:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030400, rc 0) 00000100:00100000:3.0:1769074756.223191:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf95f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:62507:x1855007616640256:12345-192.168.204.60@tcp:101:cat.0 Request processed in 227us (533us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.223197:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13211 00000100:00100000:3.0:1769074756.223215:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13212 00000100:00100000:3.0:1769074756.223221:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62512:x1855007616640512:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074756.223228:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.223236:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031600/0x7fe2323adfd52420 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adba expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.223258:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031600/0x7fe2323adfd52420 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adba expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.223270:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031600/0x7fe2323adfd52420 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adba expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.223279:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074756.223283:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074756.223285:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031600, rc 0) 00000100:00100000:3.0:1769074756.223312:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62512:x1855007616640512:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 90us (525us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.223320:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13212 00000100:00100000:0.0:1769074756.223677:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625046656:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074756.223704:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->1310719] (req 1048576->1310719) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.223720:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031400, rc 0) 00000100:00100000:0.0:1769074756.223767:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:62505:x1855007616640000:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17686us (18482us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.223775:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13593 00000100:00100000:1.0:1769074756.224163:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640384 00000100:00100000:1.0:1769074756.224171:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640384 00010000:00010000:1.0:1769074756.224222:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.224253:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6703 00000100:00100000:1.0:1769074756.224261:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616640384:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.224274:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.224293:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.224305:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.224331:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd5219c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac7f expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.224349:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.224351:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.224360:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.224370:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.224412:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.224429:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625046784:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074756.224491:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00000100:00080000:0.0:1769074756.224547:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074756.255766:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.255893:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640640 00000100:00100000:3.0:1769074756.255907:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640640 00000100:00100000:3.0:1769074756.255978:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13594 00000100:00100000:3.0:1769074756.255987:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616640640:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074756.256007:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.256026:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676031e00/0x7fe2323adfd52427 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adc1 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074756.256084:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625046912:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074756.256172:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074756.257509:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.257549:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.257645:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.258272:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640768 00000100:00100000:0.0:1769074756.258276:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640768 00000100:00100000:0.0:1769074756.258314:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616640896 00000100:00100000:0.0:1769074756.258318:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616640896 00000100:00100000:0.0:1769074756.258341:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13595 00000100:00100000:0.0:1769074756.258347:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616640768:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074756.258357:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.258376:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5242e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adc8 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.258435:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625047040:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074756.258488:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.258570:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13596 00000100:00100000:1.0:1769074756.258580:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62503:x1855007616640896:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074756.258598:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.258609:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adcf expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.258629:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 14 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.258637:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.258644:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.258646:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.258650:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2129920->2899967] (req 2129920->2899967) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.258673:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.258692:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 21 pid: 8398 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.258702:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625047168:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.258738:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 14 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.258746:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 14 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.258754:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625047296:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.258773:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3973c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.283821:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.284960:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625047168:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.285005:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625047296:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.285056:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2129920->2899967] (req 2129920->2899967) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 9 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.285084:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:1.0:1769074756.285155:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62503:x1855007616640896:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 26575us (27603us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.285165:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13596 00000100:00100000:1.0:1769074756.285218:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.285327:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.285358:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616641408 00000100:00100000:1.0:1769074756.285363:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616641408 00010000:00010000:1.0:1769074756.285389:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.285415:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616641280 00000100:00100000:1.0:1769074756.285418:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616641280 00010000:00010000:1.0:1769074756.285441:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 22 pid: 8398 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.285461:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6704 00000100:00100000:1.0:1769074756.285469:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e667c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616641408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.285483:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.285505:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.285516:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.285540:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd521b1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac94 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.285555:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.285581:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e667c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616641408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 113us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.285600:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6704 00000100:00100000:1.0:1769074756.285646:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6705 00000100:00100000:1.0:1769074756.285654:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616641280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.285662:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.285671:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 22 pid: 8398 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.285682:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 22 pid: 8398 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.285996:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd523c5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad74 expref: 22 pid: 8398 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.286007:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.286009:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.286019:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.286031:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.286157:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.286175:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625047424:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074756.286230:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00002000:00010000:3.0:1769074756.287751:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074752 -> 1769074755 00002000:00010000:3.0:1769074756.287762:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074750 -> 1769074755 00002000:00010000:3.0:1769074756.287764:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074752 -> 1769074755 00000100:00100000:3.0:1769074756.287785:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625047040:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074756.288036:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5242e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adc8 expref: 13 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.288059:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5242e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adc8 expref: 13 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.288072:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff2800, rc 0) 00000100:00100000:3.0:1769074756.288134:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616640768:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 29786us (30633us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074756.288145:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13595 00002000:00010000:0.0:1769074756.296398:0:10284:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074752 -> 1769074755 00002000:00010000:0.0:1769074756.296410:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074750 -> 1769074755 00002000:00010000:0.0:1769074756.296413:0:10284:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074752 -> 1769074755 00000100:00100000:0.0:1769074756.296450:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625046912:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074756.296491:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676031e00/0x7fe2323adfd52427 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adc1 expref: 13 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.296513:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031e00/0x7fe2323adfd52427 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adc1 expref: 13 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.296526:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031e00, rc 0) 00000100:00100000:1.0:1769074756.296669:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616640640:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 40681us (40912us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074756.296679:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13594 00000100:00100000:3.0:1769074756.301278:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.301357:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.301495:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625047424:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074756.301518:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.301569:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616641280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 15914us (16240us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.301579:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6705 00000100:00100000:1.0:1769074756.301778:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8243 00000100:00100000:1.0:1769074756.301787:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616641664:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074756.301874:0:24030:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616641664:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 88us (516us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074756.302552:0:24030:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8243 00002000:00100000:1.0:1769074756.302562:0:24030:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2129920->2134015) req@ffff89b54441ce00 x1855007616641664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/4536 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.302570:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ce00 with x1855007616641664 ext(2129920->2134015) 00002000:00010000:1.0:1769074756.302576:0:24030:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441ce00 00000100:00100000:3.0:1769074756.311167:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.313004:0:24030:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616641792 00000100:00100000:1.0:1769074756.313022:0:24030:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616641792 00002000:00100000:1.0:1769074756.313098:0:24030:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54441ed80 x1855007616641792/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.313110:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ed80 with x1855007616641792 ext(2097152->3145727) 00002000:00010000:1.0:1769074756.313116:0:24030:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441ed80 00000100:00100000:1.0:1769074756.313157:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8244 00000100:00100000:1.0:1769074756.313166:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616641792:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074756.327721:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.327788:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.327847:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616642304 00000100:00100000:0.0:1769074756.327860:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616642304 00000100:00100000:0.0:1769074756.327931:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4595 00000100:00100000:0.0:1769074756.327944:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e9c3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:62505:x1855007616642304:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074756.328233:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e9c3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62505:x1855007616642304:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 291us (442us total) trans 8589943799 rc 0/0 00000100:00100000:0.0:1769074756.328240:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4595 00000100:00100000:0.0:1769074756.339902:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.356895:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.356926:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.357060:0:24030:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616642688 00000100:00100000:3.0:1769074756.357072:0:24030:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616642688 00002000:00100000:3.0:1769074756.357123:0:24030:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2129919) req@ffff89b545388000 x1855007616642688/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.357135:0:24030:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388000 with x1855007616642688 ext(2097152->2129919) 00002000:00010000:3.0:1769074756.357142:0:24030:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545388000 00000100:00100000:3.0:1769074756.357182:0:24030:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8246 00000100:00100000:3.0:1769074756.357193:0:24030:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616642688:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.357472:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616642560 00000100:00100000:0.0:1769074756.357477:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616642560 00010000:00010000:0.0:1769074756.357499:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad97 expref: 11 pid: 6514 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.357537:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6706 00000100:00100000:0.0:1769074756.357545:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616642560:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.357556:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.357563:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad97 expref: 11 pid: 6514 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.357576:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd523fd lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad97 expref: 11 pid: 6514 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.357614:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074756.357619:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x8 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.357629:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.357637:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.357658:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.357692:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616642560:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 147us (808us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.357699:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6706 00000100:00100000:0.0:1769074756.362375:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.362531:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616642944 00000100:00100000:3.0:1769074756.362543:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616642944 00000100:00100000:3.0:1769074756.362621:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4596 00000100:00100000:3.0:1769074756.362631:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62503:x1855007616642944:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074756.362889:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62503:x1855007616642944:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 259us (520us total) trans 8589943801 rc 0/0 00000100:00100000:3.0:1769074756.362898:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4596 00000100:00100000:0.0:1769074756.364802:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.364936:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643072 00000100:00100000:1.0:1769074756.364946:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643072 00000100:00100000:1.0:1769074756.365038:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13597 00000100:00100000:1.0:1769074756.365048:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62507:x1855007616643072:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.365065:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.365083:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879addd expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.365111:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 7 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.365119:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074756.365121:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074756.365125:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 22 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.365161:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 7 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.365174:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 7 pid: 27844 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.365188:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625047680:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.365292:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074756.368907:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625047680:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.368962:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 22 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.368993:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:3.0:1769074756.369076:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62507:x1855007616643072:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4029us (4278us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.369087:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13597 00000100:00100000:0.0:1769074756.373331:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643328 00000100:00100000:0.0:1769074756.373346:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643328 00000100:00100000:0.0:1769074756.373418:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13214 00000100:00100000:0.0:1769074756.373429:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62512:x1855007616643328:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074756.373469:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.373491:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52458 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ade4 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.373610:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.373627:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879add6 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.373635:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.373670:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879add6 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.373681:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879add6 expref: 11 pid: 6513 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.373694:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625047808:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074756.373793:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6412546c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.374084:0:8402:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643200 00000100:00100000:1.0:1769074756.374092:0:8402:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643200 00002000:00100000:1.0:1769074756.374137:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2129920->2899967) req@ffff89b542faa680 x1855007616643200/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.374148:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542faa680 with x1855007616643200 ext(2129920->2899967) 00010000:00010000:1.0:1769074756.374157:0:8402:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 8 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.374169:0:8402:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 8 pid: 27844 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074756.374177:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b542faa680 00000100:00100000:1.0:1769074756.374207:0:8402:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643456 00000100:00100000:1.0:1769074756.374211:0:8402:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643456 00002000:00100000:1.0:1769074756.374227:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2899968->3145727) req@ffff89b54475ad80 x1855007616643456/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.374234:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475ad80 with x1855007616643456 ext(2899968->3145727) 00010000:00010000:1.0:1769074756.374239:0:8402:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 9 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.374247:0:8402:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 9 pid: 27844 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074756.374253:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54475ad80 00000100:00100000:1.0:1769074756.374275:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 665 00000100:00100000:1.0:1769074756.374283:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616643200:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074756.374470:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 666 00000100:00100000:1.0:1769074756.374479:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616643456:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:1.0:1769074756.396609:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52443 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879add6 expref: 11 pid: 6513 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.396629:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625047808:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.396651:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.396663:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.396668:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074756.396734:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074756.396739:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074756.396761:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.396795:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52458 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ade4 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.397051:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ade4 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.397060:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.397063:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1769074756.397067:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c600, rc 0) 00000100:00100000:1.0:1769074756.397118:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62512:x1855007616643328:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 23690us (23900us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.397126:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13214 00000100:00100000:1.0:1769074756.406072:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.406100:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.406155:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.406200:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643840 00000100:00100000:1.0:1769074756.406213:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643840 00000100:00100000:1.0:1769074756.406281:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643712 00000100:00100000:1.0:1769074756.406285:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643712 00000100:00100000:1.0:1769074756.406308:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616643968 00000100:00100000:1.0:1769074756.406311:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616643968 00000100:00100000:1.0:1769074756.406336:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13598 00000100:00100000:1.0:1769074756.406348:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616643840:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.406373:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.406397:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52466 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adf2 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.406483:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625047936:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074756.406559:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.406593:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13599 00000100:00100000:1.0:1769074756.406601:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616643712:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.406611:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.406622:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd5246d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adeb expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.406654:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625048064:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074756.406680:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.406699:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13600 00000100:00100000:1.0:1769074756.406705:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616643968:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.406712:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.406720:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd52474 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879adf9 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.406751:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625048192:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074756.406777:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.407153:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616643456:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32673us (33898us total) trans 0 rc 245760/245760 00000100:00100000:0.0:1769074756.407165:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 666 00002000:00100000:0.0:1769074756.407176:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2899968->3145727) req@ffff89b54475ad80 x1855007616643456/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:H/600/0 rc 245760/245760 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.407186:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475ad80 with x1855007616643456 ext(2899968->3145727) 00010000:00010000:0.0:1769074756.407196:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.407210:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00002000:00010000:0.0:1769074756.407217:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54475ad80 00000100:00100000:3.0:1769074756.410180:0:8402:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616643200:12345-192.168.204.60@tcp:4:cat.0 Request processed in 35888us (37024us total) trans 17179874459 rc 0/0 00000100:00100000:3.0:1769074756.410195:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 665 00002000:00100000:3.0:1769074756.410208:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2129920->2899967) req@ffff89b542faa680 x1855007616643200/t17179874459(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/448 e 0 to 0 dl 1769074767 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.410218:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542faa680 with x1855007616643200 ext(2129920->2899967) 00010000:00010000:3.0:1769074756.410228:0:8402:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.410425:0:8402:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00002000:00010000:3.0:1769074756.410444:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b542faa680 00000100:00100000:3.0:1769074756.412535:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625047936:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074756.412576:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52466 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adf2 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.412606:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52466 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adf2 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.412625:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640031200, rc 0) 00000100:00100000:3.0:1769074756.412684:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616643840:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6337us (6624us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074756.412699:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13598 00002000:00010000:3.0:1769074756.413047:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074752 -> 1769074755 00002000:00010000:3.0:1769074756.413052:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074751 -> 1769074755 00002000:00010000:3.0:1769074756.413054:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074752 -> 1769074755 00000100:00100000:3.0:1769074756.413066:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625048192:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074756.413085:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd52474 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adf9 expref: 15 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.413102:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd52474 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adf9 expref: 15 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.413111:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718000, rc 0) 00000100:00100000:3.0:1769074756.413137:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616643968:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6431us (6982us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074756.413143:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13600 00000100:00100000:3.0:1769074756.415120:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.415148:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616644096 00000100:00100000:3.0:1769074756.415157:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616644096 00000100:00100000:3.0:1769074756.415200:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13601 00000100:00100000:3.0:1769074756.415210:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62509:x1855007616644096:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074756.415226:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.415245:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae00 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.415268:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.415271:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.415323:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.415340:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.415352:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164800, rc 0) 00000100:00100000:3.0:1769074756.415408:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62509:x1855007616644096:12345-192.168.204.60@tcp:101:cat.0 Request processed in 199us (297us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.415415:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13601 00010000:00010000:0.0:1769074756.421031:0:36516:0:(ldlm_lockd.c:783:ldlm_cb_interpret()) ### lost race - client has a lock but no inode ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 643072->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad7b expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.421067:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625048064:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074756.421102:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd5246d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adeb expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.421125:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd5246d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adeb expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.421139:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640030a00, rc 0) 00000100:00100000:0.0:1769074756.421217:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616643712:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 14616us (15116us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074756.421227:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13599 00000100:00100000:3.0:1769074756.435431:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.435606:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616644480 00000100:00100000:1.0:1769074756.435617:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616644480 00000100:00100000:1.0:1769074756.435682:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13215 00000100:00100000:1.0:1769074756.435693:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62512:x1855007616644480:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.435713:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.435735:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52482 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae07 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.435868:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.435899:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.435911:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074756.435924:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074756.436026:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52482 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae07 expref: 13 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.436300:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae07 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.436311:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.436314:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074756.436318:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b719800, rc 0) 00000100:00100000:1.0:1769074756.436375:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62512:x1855007616644480:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 684us (955us total) trans 8589943802 rc 0/0 00000100:00100000:1.0:1769074756.436385:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13215 00000100:00100000:3.0:1769074756.437169:0:8402:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616644608 00000100:00100000:3.0:1769074756.437218:0:8402:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616644608 00002000:00100000:3.0:1769074756.437261:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b544759c00 x1855007616644608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.437270:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759c00 with x1855007616644608 ext(2097152->3145727) 00002000:00010000:3.0:1769074756.437276:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b544759c00 00000100:00100000:3.0:1769074756.437307:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8247 00000100:00100000:3.0:1769074756.437316:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616644608:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.438621:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616644224 00000100:00100000:0.0:1769074756.438632:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616644224 00010000:00010000:0.0:1769074756.438669:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.438700:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6708 00000100:00100000:0.0:1769074756.438709:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616644224:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.438723:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.438746:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.438761:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.438787:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd52435 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2129920->2899967) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adcf expref: 10 pid: 27844 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.438804:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.438831:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.438839:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.438850:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.439369:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.439386:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625048320:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074756.440387:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6412546c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.444915:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616644864 00000100:00100000:1.0:1769074756.445065:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616645120 00000100:00100000:1.0:1769074756.445070:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616645120 00000100:00100000:1.0:1769074756.445096:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616645376 00000100:00100000:1.0:1769074756.445099:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616645376 00000100:00100000:1.0:1769074756.445118:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13605 00000100:00100000:1.0:1769074756.445125:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2406:x1855007616645120:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074756.445147:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445161:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.445169:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445175:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.445177:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.445179:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 5/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445202:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445212:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.445227:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625048832:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.445285:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.445295:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.445304:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625048960:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.445329:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.445348:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13606 00000100:00100000:1.0:1769074756.445356:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616645376:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074756.445380:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445391:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.445399:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445406:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445412:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.445414:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.445416:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 6/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445435:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445442:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 14 pid: 10284 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.445451:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625049088:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.445483:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.445492:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 9 pid: 28955 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.445501:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625049216:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.445529:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.445538:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.445547:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625049344:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.445565:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.447202:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616644864 00000100:00100000:1.0:1769074756.447351:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13607 00000100:00100000:1.0:1769074756.447361:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2406:x1855007616644864:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074756.447410:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd524ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.447431:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.447437:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074756.447439:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074756.447443:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd524ba lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.447473:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 14 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.447485:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 14 pid: 10284 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.447498:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa22680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625049472:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.447577:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074756.458443:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625048320:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074756.458471:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.458528:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616644224:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 19819us (21791us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.458542:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6708 00000100:00100000:3.0:1769074756.467545:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.467575:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.467734:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625048704:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.467772:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.467878:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625048576:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074756.467895:0:28954:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.467936:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616645504 00000100:00100000:0.0:1769074756.467944:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616645504 00010000:00010000:0.0:1769074756.467984:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 643072->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad7b expref: 22 pid: 28955 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.468022:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616645632 00000100:00100000:0.0:1769074756.468026:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616645632 00010000:00010000:0.0:1769074756.468039:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd521a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac86 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074756.468064:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6709 00000100:00100000:0.0:1769074756.468073:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616645504:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.468090:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.468110:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 643072->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad7b expref: 22 pid: 28955 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.468122:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd523be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 643072->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad7b expref: 22 pid: 28955 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.468152:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.468154:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.468159:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd52497 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468169:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd52497 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468189:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.468237:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616645504:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 165us (701us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.468246:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6709 00000100:00100000:0.0:1769074756.468266:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6710 00000100:00100000:0.0:1769074756.468273:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616645632:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.468282:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.468296:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd521a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac86 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.468306:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd521a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac86 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.468321:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd521a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac86 expref: 15 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.468338:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.468340:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.468343:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468350:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468369:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.468391:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2406:x1855007616645632:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 118us (815us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.468397:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6710 00010000:00010000:0.0:1769074756.468419:0:28954:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468427:0:28954:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468450:0:28954:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468457:0:28954:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468465:0:28954:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468472:0:28954:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b437e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074756.468477:0:28954:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468490:0:28954:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468500:0:28954:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.468506:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd5249e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074756.468554:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616644992:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 24426us (24849us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.468562:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13603 00000100:00100000:3.0:1769074756.474350:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.474407:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.474526:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625049216:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.474541:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625049344:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.474565:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616645888 00000100:00100000:1.0:1769074756.474572:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616645888 00002000:00100000:1.0:1769074756.474640:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1310719) req@ffff89b54441fb80 x1855007616645888/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.474650:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441fb80 with x1855007616645888 ext(1048576->1310719) 00010000:00010000:1.0:1769074756.474659:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 15 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.474684:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 15 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074756.474692:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441fb80 00000100:00100000:1.0:1769074756.474723:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 667 00000100:00100000:1.0:1769074756.474733:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616645888:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074756.475068:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616645760 00000100:00100000:0.0:1769074756.475074:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616645760 00010000:00010000:0.0:1769074756.475095:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074756.475118:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6711 00000100:00100000:0.0:1769074756.475125:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616645760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.475137:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.475155:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.475167:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.475199:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd521b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac9b expref: 13 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.475217:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.475258:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616645760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 133us (924us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.475266:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6711 00000100:00100000:3.0:1769074756.477157:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.477253:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625048832:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.477265:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625048960:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.477290:0:9048:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074756.478568:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.484320:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.484349:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646272 00000100:00100000:1.0:1769074756.484360:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646272 00010000:00010000:1.0:1769074756.484424:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.484477:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6712 00000100:00100000:1.0:1769074756.484488:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616646272:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.484506:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.484533:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.484547:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.484573:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641520e00/0x7fe2323adfd521aa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ac8d expref: 14 pid: 28519 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.484608:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.484668:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616646272:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 181us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.484677:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6712 00000100:00100000:1.0:1769074756.484787:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.484841:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646144 00000100:00100000:1.0:1769074756.484847:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646144 00010000:00010000:1.0:1769074756.484863:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.484885:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6713 00000100:00100000:1.0:1769074756.484892:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616646144:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.484904:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.484913:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.484922:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.484936:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b682164000/0x7fe2323adfd523e1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad90 expref: 9 pid: 28955 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.484945:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.484947:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.484953:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.484964:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.484991:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.485012:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2407:x1855007616646144:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 120us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.485019:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6713 00010000:00010000:1.0:1769074756.485044:0:9048:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485053:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485079:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485088:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485096:0:9048:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485103:0:9048:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f70400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074756.485107:0:9048:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485116:0:9048:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485128:0:9048:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.485135:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd524ac lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.485183:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616645120:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 40058us (40679us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.485190:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13605 00000100:00100000:3.0:1769074756.486077:0:8402:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616644608:12345-192.168.204.60@tcp:3:cat.0 Request processed in 48757us (49003us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074756.486086:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8247 00002000:00100000:3.0:1769074756.486096:0:8402:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b544759c00 x1855007616644608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.486106:0:8402:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759c00 with x1855007616644608 ext(2097152->3145727) 00010000:00010000:3.0:1769074756.486115:0:8402:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 8 pid: 28955 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.486126:0:8402:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd5247b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae00 expref: 8 pid: 28955 timeout: 1844 lvb_type: 0 00002000:00010000:3.0:1769074756.486133:0:8402:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b544759c00 00000100:00100000:0.0:1769074756.487756:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646016 00000100:00100000:0.0:1769074756.487769:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646016 00002000:00100000:1.0:1769074756.487959:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54441ca80 x1855007616646016/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.487970:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ca80 with x1855007616646016 ext(3145728->4194303) 00010000:00010000:1.0:1769074756.487978:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.487991:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074756.487998:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54441ca80 00000100:00100000:1.0:1769074756.488031:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646400 00000100:00100000:1.0:1769074756.488035:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646400 00002000:00100000:1.0:1769074756.488052:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54441f100 x1855007616646400/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.488060:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f100 with x1855007616646400 ext(3145728->4194303) 00010000:00010000:1.0:1769074756.488065:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 17 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.488074:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd5240b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ada5 expref: 17 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074756.488081:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441f100 00000100:00100000:1.0:1769074756.488105:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 668 00000100:00100000:1.0:1769074756.488113:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616646016:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074756.502782:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616645888:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 28030us (28363us total) trans 21474841135 rc 0/0 00000100:00100000:0.0:1769074756.502800:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 667 00002000:00100000:0.0:1769074756.502845:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1310719) req@ffff89b54441fb80 x1855007616645888/t21474841135(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/448 e 0 to 0 dl 1769074767 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.502856:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441fb80 with x1855007616645888 ext(1048576->1310719) 00010000:00010000:0.0:1769074756.502866:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 19 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.502883:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 19 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:0.0:1769074756.502891:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441fb80 00000100:00100000:1.0:1769074756.512196:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.512342:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646528 00000100:00100000:0.0:1769074756.512353:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646528 00010000:00010000:0.0:1769074756.512394:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 19 pid: 10284 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.512439:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6714 00000100:00100000:0.0:1769074756.512449:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616646528:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.512466:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.512491:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 19 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.512502:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 19 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.512525:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b676031400/0x7fe2323adfd52412 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1310719) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adac expref: 19 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.512541:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.512543:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.512547:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd524ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.512556:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd524ba lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.512594:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.512644:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616646528:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 197us (459us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.512651:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6714 00000100:00100000:3.0:1769074756.513970:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616646016:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 25856us (36818us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074756.513980:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 668 00002000:00100000:3.0:1769074756.513989:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54441ca80 x1855007616646016/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.513998:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ca80 with x1855007616646016 ext(3145728->4194303) 00010000:00010000:3.0:1769074756.514005:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.514020:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd52451 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879addd expref: 21 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:3.0:1769074756.514027:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54441ca80 00000100:00100000:1.0:1769074756.516514:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.516936:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646784 00000100:00100000:0.0:1769074756.516947:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646784 00000100:00100000:0.0:1769074756.517002:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13216 00000100:00100000:0.0:1769074756.517012:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62516:x1855007616646784:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074756.517037:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.517058:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd524c1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae0e expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.517139:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517157:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517171:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517186:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517192:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517199:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517324:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517337:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae07 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.517346:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074756.517384:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae07 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.517404:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae07 expref: 11 pid: 6513 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.517419:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625049600:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074756.517499:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.524730:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.524802:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.526680:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616646912 00000100:00100000:3.0:1769074756.526695:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616646912 00000100:00100000:3.0:1769074756.526745:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13608 00000100:00100000:3.0:1769074756.526754:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62509:x1855007616646912:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074756.526771:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.526788:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae15 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.526839:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.527032:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.527039:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074756.527044:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.530921:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.530964:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.533337:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:0.0:1769074756.533698:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719800/0x7fe2323adfd52489 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae07 expref: 11 pid: 6513 timeout: 1844 lvb_type: 0 00000100:00100000:0.0:1769074756.533717:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625049600:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074756.533741:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.533835:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.533843:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074756.533864:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074756.533993:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534001:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534009:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534015:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910ca00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074756.534020:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534030:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534047:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.534050:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074756.534055:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 2/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534061:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd524d6 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534086:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534092:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534097:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534102:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534107:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074756.534109:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534114:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534122:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074756.534124:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074756.534127:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534132:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c400/0x7fe2323adfd524c8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534141:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534146:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534152:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534157:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910de00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074756.534160:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534167:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534175:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47f00) 00010000:00010000:1.0:1769074756.534178:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074756.534181:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534188:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd524cf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074756.534248:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd524c1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae0e expref: 224 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.534260:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074756.534263:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074756.534266:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd524c1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae0e expref: 224 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.534275:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:1.0:1769074756.534364:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62516:x1855007616646784:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17353us (17867us total) trans 8589943803 rc 301/301 00000100:00100000:1.0:1769074756.534374:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13216 00010000:00010000:1.0:1769074756.541245:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541270:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541314:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541322:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541330:0:28955:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541338:0:28955:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074756.541342:0:28955:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541350:0:28955:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd524a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000020 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541373:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.541375:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.541387:0:28955:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541562:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.541636:0:28955:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.541653:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625049728:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074756.541761:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074756.545519:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074756.547234:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547249:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547278:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547286:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547295:0:27843:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547303:0:27843:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f71a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074756.547307:0:27843:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547316:0:27843:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd524b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000020 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547337:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.547339:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.547351:0:27843:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547361:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.547550:0:27843:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.547568:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625049856:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074756.547734:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.555752:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.555848:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616647680 00000100:00100000:3.0:1769074756.555863:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616647680 00002000:00100000:3.0:1769074756.556442:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54d1dc000 x1855007616647680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.556470:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dc000 with x1855007616647680 ext(3145728->4194303) 00002000:00010000:3.0:1769074756.556477:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dc000 00000100:00100000:3.0:1769074756.556514:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8248 00000100:00100000:3.0:1769074756.556525:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616647680:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074756.563467:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.563702:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616647808 00000100:00100000:3.0:1769074756.563717:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616647808 00002000:00100000:3.0:1769074756.563782:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54d1dce00 x1855007616647808/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.563795:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dce00 with x1855007616647808 ext(3145728->4194303) 00002000:00010000:3.0:1769074756.563802:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1dce00 00000100:00100000:3.0:1769074756.563860:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8249 00000100:00100000:3.0:1769074756.563871:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616647808:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074756.579748:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616647680:12345-192.168.204.60@tcp:3:cat.0 Request processed in 23215us (24003us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074756.579763:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8248 00002000:00100000:0.0:1769074756.579776:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54d1dc000 x1855007616647680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.579787:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dc000 with x1855007616647680 ext(3145728->4194303) 00002000:00010000:0.0:1769074756.579793:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dc000 00000100:00100000:1.0:1769074756.582228:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616648064 00000100:00100000:1.0:1769074756.582243:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616648064 00000100:00100000:1.0:1769074756.582321:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13217 00000100:00100000:1.0:1769074756.582332:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62512:x1855007616648064:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:1.0:1769074756.582405:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.582434:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031600/0x7fe2323adfd52420 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adba expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.582454:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.582459:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.582542:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62512:x1855007616648064:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 211us (4330us total) trans 0 rc -116/-116 00000100:00100000:1.0:1769074756.582549:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13217 00000100:00100000:0.0:1769074756.583752:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616647808:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 19877us (20296us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074756.583765:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8249 00002000:00100000:0.0:1769074756.583778:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54d1dce00 x1855007616647808/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.583789:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dce00 with x1855007616647808 ext(3145728->4194303) 00002000:00010000:0.0:1769074756.583796:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1dce00 00000100:00100000:3.0:1769074756.590789:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.591519:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616648448 00000100:00100000:3.0:1769074756.591532:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616648448 00000100:00100000:3.0:1769074756.591608:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13610 00000100:00100000:3.0:1769074756.591619:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62509:x1855007616648448:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074756.591643:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.591664:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd524f2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae23 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.591696:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.591698:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.591709:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd524f2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae23 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.591728:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd524f2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae23 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.591743:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff2800, rc 0) 00000100:00100000:3.0:1769074756.591848:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62509:x1855007616648448:12345-192.168.204.60@tcp:101:cat.0 Request processed in 231us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.591856:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13610 00000100:00100000:1.0:1769074756.592561:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616648320 00000100:00100000:1.0:1769074756.592570:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616648320 00000100:00100000:1.0:1769074756.592673:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4597 00000100:00100000:1.0:1769074756.592683:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62512:x1855007616648320:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074756.592971:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62512:x1855007616648320:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 289us (2195us total) trans 8589943804 rc 0/0 00000100:00100000:1.0:1769074756.592979:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4597 00000100:00100000:1.0:1769074756.606231:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616649344 00000100:00100000:1.0:1769074756.606246:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616649344 00000100:00100000:1.0:1769074756.606362:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4598 00000100:00100000:1.0:1769074756.606374:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62512:x1855007616649344:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074756.606698:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62512:x1855007616649344:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 326us (786us total) trans 8589943805 rc 0/0 00000100:00100000:1.0:1769074756.606706:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4598 00000100:00100000:0.0:1769074756.619796:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.620068:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616649472 00000100:00100000:3.0:1769074756.620081:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616649472 00002000:00100000:3.0:1769074756.620136:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b546f94a80 x1855007616649472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.620150:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94a80 with x1855007616649472 ext(3145728->4194303) 00002000:00010000:3.0:1769074756.620159:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f94a80 00000100:00100000:3.0:1769074756.620197:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8250 00000100:00100000:3.0:1769074756.620208:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2409:x1855007616649472:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.637601:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.647335:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2409:x1855007616649472:12345-192.168.204.60@tcp:3:cat.0 Request processed in 27115us (27536us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074756.647350:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8250 00002000:00100000:3.0:1769074756.647364:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b546f94a80 x1855007616649472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.647376:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94a80 with x1855007616649472 ext(3145728->4194303) 00002000:00010000:3.0:1769074756.647383:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f94a80 00000100:00100000:3.0:1769074756.648871:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616649856 00000100:00100000:3.0:1769074756.648884:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616649856 00000100:00100000:3.0:1769074756.654373:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616649600:12345-192.168.204.60@tcp:3:cat.0 Request processed in 15585us (16781us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074756.654388:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8251 00002000:00100000:3.0:1769074756.654410:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f94000 x1855007616649600/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.654420:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94000 with x1855007616649600 ext(4194304->4808703) 00002000:00010000:3.0:1769074756.654427:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94000 00000100:00100000:0.0:1769074756.659219:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.659250:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.659281:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.659301:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.659331:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616649984 00000100:00100000:0.0:1769074756.659343:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616649984 00000100:00100000:0.0:1769074756.659499:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616650240 00000100:00100000:0.0:1769074756.659504:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616650240 00000100:00100000:0.0:1769074756.659532:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616650112 00000100:00100000:0.0:1769074756.659536:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616650112 00000100:00100000:0.0:1769074756.659550:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616650368 00000100:00100000:0.0:1769074756.659552:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616650368 00000100:00100000:0.0:1769074756.659571:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13616 00000100:00100000:0.0:1769074756.659591:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616649984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074756.659610:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.659628:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae54 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.659650:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.659652:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.659662:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.659683:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.659696:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:0.0:1769074756.659750:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616649984:12345-192.168.204.60@tcp:101:cat.0 Request processed in 161us (542us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.659758:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13616 00000100:00100000:0.0:1769074756.659776:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13617 00000100:00100000:0.0:1769074756.659782:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616650240:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074756.659789:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.659799:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae62 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.659833:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.659835:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.659840:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.659853:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.659862:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:0.0:1769074756.659885:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616650240:12345-192.168.204.60@tcp:101:cat.0 Request processed in 103us (634us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.659892:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13617 00000100:00100000:0.0:1769074756.659905:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13618 00000100:00100000:0.0:1769074756.659911:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616650112:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074756.659916:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1769074756.661114:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13619 00000100:00100000:1.0:1769074756.661135:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616650368:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.661148:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.661164:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae69 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.661184:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.661186:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.661193:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae69 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.661206:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae69 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.661217:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867000, rc 0) 00000100:00100000:1.0:1769074756.661261:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616650368:12345-192.168.204.60@tcp:101:cat.0 Request processed in 136us (1958us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.661267:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13619 00010000:00010000:0.0:1769074756.664692:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae5b expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.664744:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.664746:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.664760:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.664780:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.664797:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:0.0:1769074756.664894:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616650112:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4981us (5610us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.664904:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13618 00000100:00100000:0.0:1769074756.669676:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.669779:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616650624 00000100:00100000:1.0:1769074756.669791:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616650624 00000100:00100000:1.0:1769074756.669900:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4599 00000100:00100000:1.0:1769074756.669911:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62507:x1855007616650624:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074756.670162:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:62507:x1855007616650624:12345-192.168.204.60@tcp:35:cat.0 Request processed in 251us (496us total) trans 8589943807 rc 0/0 00000100:00100000:1.0:1769074756.670170:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4599 00000100:00100000:0.0:1769074756.682450:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.682552:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616650752 00000100:00100000:3.0:1769074756.682566:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616650752 00002000:00100000:3.0:1769074756.682639:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f95880 x1855007616650752/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.682651:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95880 with x1855007616650752 ext(4194304->4808703) 00002000:00010000:3.0:1769074756.682656:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f95880 00000100:00100000:3.0:1769074756.682693:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8252 00000100:00100000:3.0:1769074756.682704:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616650752:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074756.717974:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616650752:12345-192.168.204.60@tcp:3:cat.0 Request processed in 35260us (35539us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074756.717989:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8252 00002000:00100000:3.0:1769074756.718002:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f95880 x1855007616650752/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.718013:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95880 with x1855007616650752 ext(4194304->4808703) 00002000:00010000:3.0:1769074756.718019:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f95880 00000100:00100000:0.0:1769074756.727020:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.727177:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651008 00000100:00100000:3.0:1769074756.727190:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651008 00000100:00100000:3.0:1769074756.727304:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4600 00000100:00100000:3.0:1769074756.727316:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62509:x1855007616651008:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074756.727668:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62509:x1855007616651008:12345-192.168.204.60@tcp:35:cat.0 Request processed in 352us (656us total) trans 8589943808 rc 0/0 00000100:00100000:3.0:1769074756.727677:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4600 00000100:00100000:0.0:1769074756.759240:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.773155:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651264 00000100:00100000:1.0:1769074756.773166:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651264 00000100:00100000:1.0:1769074756.773221:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651392 00000100:00100000:1.0:1769074756.773225:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651392 00000100:00100000:1.0:1769074756.773245:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651520 00000100:00100000:1.0:1769074756.773248:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651520 00000100:00100000:1.0:1769074756.773264:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651648 00000100:00100000:1.0:1769074756.773267:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651648 00000100:00100000:1.0:1769074756.773287:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13620 00000100:00100000:1.0:1769074756.773295:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616651264:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.773315:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.773334:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae77 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.773358:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773360:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773369:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773386:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773399:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866800, rc 0) 00000100:00100000:1.0:1769074756.773454:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616651264:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 160us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.773462:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13620 00000100:00100000:1.0:1769074756.773482:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13621 00000100:00100000:1.0:1769074756.773488:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616651392:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.773495:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.773504:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae7e expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.773518:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773520:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773525:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773535:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773549:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866600, rc 0) 00000100:00100000:1.0:1769074756.773569:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616651392:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 81us (574us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.773586:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13621 00000100:00100000:1.0:1769074756.773600:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13622 00000100:00100000:1.0:1769074756.773606:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616651520:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.773613:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.773620:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae85 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.773631:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773633:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773637:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773648:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773656:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:1.0:1769074756.773676:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616651520:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 71us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.773682:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13622 00000100:00100000:1.0:1769074756.773694:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13623 00000100:00100000:1.0:1769074756.773700:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616651648:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.773709:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.773716:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae8c expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.773727:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773729:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074756.773733:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae8c expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773741:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae8c expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.773748:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:1.0:1769074756.773767:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616651648:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 68us (697us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.773772:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13623 00000100:00100000:0.0:1769074756.780951:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.781006:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.784601:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651776 00000100:00100000:3.0:1769074756.784620:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651776 00000100:00100000:3.0:1769074756.784696:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616651904 00000100:00100000:3.0:1769074756.784701:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616651904 00000100:00100000:3.0:1769074756.784747:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13624 00000100:00100000:3.0:1769074756.784758:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616651776:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074756.784783:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.784825:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52593 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae93 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.784857:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.784859:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074756.784870:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52593 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae93 expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.784888:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52593 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae93 expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074756.784902:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3e00, rc 0) 00000100:00100000:0.0:1769074756.785086:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13625 00000100:00100000:0.0:1769074756.785100:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616651904:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074756.785109:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.785121:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ae9a expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.785137:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.785139:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.785145:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae9a expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.785156:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae9a expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.785165:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:0.0:1769074756.785236:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616651904:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 141us (4228us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.785242:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13625 00000100:00100000:3.0:1769074756.790864:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616651776:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6093us (9913us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.790878:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13624 00000100:00100000:0.0:1769074756.806548:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.812734:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.812782:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.812861:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.812956:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616652288 00000100:00100000:1.0:1769074756.812967:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616652288 00010000:00010000:1.0:1769074756.813008:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd52570 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae70 expref: 15 pid: 6512 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.813048:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616652544 00000100:00100000:1.0:1769074756.813051:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616652544 00010000:00010000:1.0:1769074756.813063:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ade4 expref: 16 pid: 6513 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.813082:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616652416 00000100:00100000:1.0:1769074756.813085:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616652416 00010000:00010000:1.0:1769074756.813098:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52531 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae4d expref: 13 pid: 6512 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.813119:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6719 00000100:00100000:1.0:1769074756.813127:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616652288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.813143:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.813150:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd52570 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae70 expref: 16 pid: 6512 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813164:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd52570 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae70 expref: 16 pid: 6512 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813190:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd52570 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae70 expref: 16 pid: 6512 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813210:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.813214:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813223:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813230:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.813281:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616652288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 154us (556us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.813289:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6719 00000100:00100000:1.0:1769074756.813307:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6720 00000100:00100000:1.0:1769074756.813313:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2409:x1855007616652544:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.813323:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.813328:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ade4 expref: 14 pid: 6513 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813336:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ade4 expref: 14 pid: 6513 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813349:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5245f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ade4 expref: 14 pid: 6513 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813357:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.813361:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813367:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813371:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.813390:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616652544:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 77us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.813397:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6720 00000100:00100000:1.0:1769074756.813410:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6721 00000100:00100000:1.0:1769074756.813416:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616652416:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.813423:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.813426:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52531 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae4d expref: 13 pid: 6512 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813435:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52531 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae4d expref: 13 pid: 6512 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813451:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52531 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae4d expref: 13 pid: 6512 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074756.813460:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.813463:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813470:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813477:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813500:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.813520:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616652416:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 104us (659us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.813527:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6721 00010000:00010000:1.0:1769074756.813559:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.813566:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074756.813606:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074756.813736:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea1 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.813753:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea1 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.813763:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.813766:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074756.813768:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5200, rc 0) 00000100:00100000:1.0:1769074756.813800:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62522:x1855007616652160:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 7024us (7263us total) trans 8589943809 rc 0/0 00000100:00100000:1.0:1769074756.813856:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13220 00000100:00100000:0.0:1769074756.823506:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.823619:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616652672 00000100:00100000:1.0:1769074756.823632:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616652672 00000100:00100000:1.0:1769074756.823708:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13221 00000100:00100000:1.0:1769074756.823720:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62522:x1855007616652672:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074756.823755:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.823783:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525b6 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aea8 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.823939:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.823955:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea1 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.823963:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.824001:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea1 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.824014:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd525a8 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea1 expref: 14 pid: 6512 timeout: 1844 lvb_type: 0 00000100:00100000:1.0:1769074756.824032:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625050368:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074756.824105:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.834184:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625050368:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074756.834219:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:0.0:1769074756.835623:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074756.837790:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.837837:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074756.837896:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074756.837900:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074756.837935:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.837973:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525b6 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.837991:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.838000:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.838003:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074756.838006:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:1.0:1769074756.838068:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62522:x1855007616652672:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 14349us (14572us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.838080:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13221 00000100:00100000:0.0:1769074756.845605:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.845737:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616653056 00000100:00100000:1.0:1769074756.845751:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616653056 00000100:00100000:0.0:1769074756.870031:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.874190:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.874286:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616653440 00000100:00100000:3.0:1769074756.874298:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616653440 00000100:00100000:3.0:1769074756.874358:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4601 00000100:00100000:3.0:1769074756.874367:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e9c2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62522:x1855007616653440:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074756.874651:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e9c2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62522:x1855007616653440:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 283us (471us total) trans 8589943810 rc 0/0 00000100:00100000:3.0:1769074756.874659:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4601 00000100:00100000:1.0:1769074756.889612:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.889651:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.889692:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.889758:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616653568 00000100:00100000:3.0:1769074756.889770:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616653568 00010000:00010000:3.0:1769074756.889850:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae9a expref: 13 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074756.889891:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616653696 00000100:00100000:3.0:1769074756.889897:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616653696 00010000:00010000:3.0:1769074756.889929:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 18 pid: 36516 timeout: 1844 lvb_type: 0 00000100:00100000:3.0:1769074756.889952:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616653824 00000100:00100000:3.0:1769074756.889957:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616653824 00010000:00010000:3.0:1769074756.889973:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd524f2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae23 expref: 12 pid: 27843 timeout: 1844 lvb_type: 0 00000100:00100000:3.0:1769074756.889994:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6723 00000100:00100000:3.0:1769074756.890004:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616653568:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.890023:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074756.890050:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae9a expref: 13 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:3.0:1769074756.890068:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae9a expref: 13 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:3.0:1769074756.890092:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5259a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae9a expref: 13 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:3.0:1769074756.890104:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074756.890129:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616653568:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 126us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.890136:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6723 00000100:00100000:3.0:1769074756.890155:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6724 00000100:00100000:3.0:1769074756.890161:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616653696:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.890170:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074756.890180:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 18 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.890196:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 18 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.890216:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd524e4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae1c expref: 18 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.890239:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074756.890268:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616653696:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 105us (614us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.890281:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6724 00000100:00100000:0.0:1769074756.906756:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.908087:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616653952 00000100:00100000:3.0:1769074756.908104:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616653952 00002000:00100000:3.0:1769074756.908173:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2785280->2789375) req@ffff89b5436d5500 x1855007616653952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.908185:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d5500 with x1855007616653952 ext(2785280->2789375) 00002000:00010000:3.0:1769074756.908191:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d5500 00000100:00100000:3.0:1769074756.908231:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8253 00000100:00100000:3.0:1769074756.908240:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616653952:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074756.908418:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616653952:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 178us (1672us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074756.908426:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8253 00002000:00100000:3.0:1769074756.908432:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2785280->2789375) req@ffff89b5436d5500 x1855007616653952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/4536 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.908448:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d5500 with x1855007616653952 ext(2785280->2789375) 00002000:00010000:3.0:1769074756.908452:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d5500 00000100:00100000:0.0:1769074756.912410:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.912529:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616654208 00000100:00100000:1.0:1769074756.912541:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616654208 00000100:00100000:1.0:1769074756.912619:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13223 00000100:00100000:1.0:1769074756.912631:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62526:x1855007616654208:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.912659:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.912682:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866200/0x7fe2323adfd525d9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aebd expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.912781:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.912797:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.912864:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.912896:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.912904:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.912910:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913046:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913056:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913063:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074756.913073:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074756.913160:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913168:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913173:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913179:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913184:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074756.913188:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913195:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913206:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074756.913209:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074756.913214:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913224:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd525e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913237:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913243:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913249:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913254:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074756.913257:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913261:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913268:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6510d4600) 00010000:00010000:1.0:1769074756.913270:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074756.913273:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913277:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd525e7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074756.913318:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866200/0x7fe2323adfd525d9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.913331:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.913342:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074756.913345:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074756.913347:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867a00, rc 0) 00000100:00100000:1.0:1769074756.913405:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62526:x1855007616654208:12345-192.168.204.60@tcp:101:cat.0 Request processed in 776us (1009us total) trans 8589943811 rc 0/0 00000100:00100000:1.0:1769074756.913412:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13223 00000100:00100000:1.0:1769074756.929981:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.952858:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.952973:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616654464 00000100:00100000:0.0:1769074756.952983:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616654464 00000100:00100000:0.0:1769074756.953041:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13628 00000100:00100000:0.0:1769074756.953051:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62516:x1855007616654464:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074756.953074:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074756.953093:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aecb expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.953126:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 24 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.953136:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.953145:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.953152:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.953154:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074756.953159:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [905216->1048575] (req 905216->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 24 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074756.953191:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.953202:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 12 pid: 27843 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074756.953216:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625051392:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074756.953311:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.953320:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 13 pid: 27844 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074756.953328:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625051520:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074756.953357:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 24 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074756.953367:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 24 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074756.953376:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625051648:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074756.953398:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.955288:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.955323:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.955359:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.955433:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616654592 00000100:00100000:0.0:1769074756.955441:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616654592 00010000:00010000:0.0:1769074756.955474:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd52507 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae46 expref: 25 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074756.955489:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616654848 00000100:00100000:3.0:1769074756.955496:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616654848 00000100:00100000:0.0:1769074756.955510:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616654720 00000100:00100000:0.0:1769074756.955515:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616654720 00010000:00010000:3.0:1769074756.955519:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 12 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.955533:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52593 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae93 expref: 14 pid: 27843 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074756.955553:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6726 00000100:00100000:0.0:1769074756.955566:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6727 00000100:00100000:3.0:1769074756.955567:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616654592:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1769074756.955589:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544684700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616654848:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074756.955593:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.955602:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.955620:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 12 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:3.0:1769074756.955620:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd52507 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae46 expref: 25 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:3.0:1769074756.955649:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd52507 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae46 expref: 25 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.961425:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 12 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.961473:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd524dd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae15 expref: 12 pid: 36516 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074756.961491:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.961733:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544684700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616654848:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6145us (6407us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.961745:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6727 00010000:00010000:3.0:1769074756.966582:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd52507 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae46 expref: 25 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:3.0:1769074756.966608:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.966759:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616654592:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11194us (11470us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.966770:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6726 00000100:00100000:1.0:1769074756.967595:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.967627:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616655232 00000100:00100000:1.0:1769074756.967637:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616655232 00010000:00010000:1.0:1769074756.967669:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 15 pid: 27844 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074756.967708:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6729 00000100:00100000:1.0:1769074756.967718:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616655232:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074756.967733:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074756.967755:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 15 pid: 27844 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.967773:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 15 pid: 27844 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.967795:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074756.967856:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616655232:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074756.967864:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6729 00000100:00100000:1.0:1769074756.967902:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.968672:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616655360 00000100:00100000:0.0:1769074756.968679:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616655360 00010000:00010000:0.0:1769074756.968706:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 24 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074756.968744:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6730 00000100:00100000:0.0:1769074756.968753:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616655360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074756.968762:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074756.968775:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 24 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.968786:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 24 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074756.968803:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074756.968849:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616655360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 98us (945us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074756.968856:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6730 00000100:00100000:0.0:1769074756.968915:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.969964:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625051392:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.969987:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd52577 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae77 expref: 13 pid: 27844 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074756.970001:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625051520:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.970015:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd524f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae2a expref: 23 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074756.970025:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625051648:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074756.970059:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [905216->1048575] (req 905216->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 22 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074756.970083:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d800, rc 0) 00000100:00100000:3.0:1769074756.970138:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62516:x1855007616654464:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17087us (17288us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074756.970150:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13628 00000100:00100000:3.0:1769074756.982086:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.982118:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.982158:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616655616 00000100:00100000:3.0:1769074756.982168:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616655616 00000100:00100000:0.0:1769074756.982181:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616655744 00000100:00100000:0.0:1769074756.982187:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616655744 00002000:00100000:3.0:1769074756.982217:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441c380 x1855007616655616/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.982218:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54441c700 x1855007616655744/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074756.982241:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c380 with x1855007616655616 ext(1048576->2097151) 00002000:00010000:3.0:1769074756.982247:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441c380 00000100:00100000:3.0:1769074756.982280:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8254 00000100:00100000:3.0:1769074756.982289:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616655616:12345-192.168.204.60@tcp:3:cat.0 00002000:00100000:0.0:1769074756.982328:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c700 with x1855007616655744 ext(1048576->2097151) 00002000:00010000:0.0:1769074756.982333:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441c700 00000100:00100000:0.0:1769074756.982356:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8255 00000100:00100000:0.0:1769074756.982366:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616655744:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.983230:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.983419:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.986488:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616655488 00000100:00100000:1.0:1769074756.986505:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616655488 00002000:00100000:1.0:1769074756.986558:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5497a9f80 x1855007616655488/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.986579:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9f80 with x1855007616655488 ext(0->1048575) 00010000:00010000:1.0:1769074756.986590:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 11 pid: 27843 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074756.986607:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 11 pid: 27843 timeout: 1844 lvb_type: 1 00002000:00010000:1.0:1769074756.986615:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a9f80 00000100:00100000:1.0:1769074756.986639:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616656000 00000100:00100000:1.0:1769074756.986643:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616656000 00002000:00100000:1.0:1769074756.986661:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3444736->3448831) req@ffff89b5497aad80 x1855007616656000/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074756.986669:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aad80 with x1855007616656000 ext(3444736->3448831) 00002000:00010000:1.0:1769074756.986674:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aad80 00000100:00100000:1.0:1769074756.986696:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 670 00000100:00100000:1.0:1769074756.986705:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616655488:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.987768:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8256 00000100:00100000:0.0:1769074756.987783:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616656000:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.987980:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616656000:12345-192.168.204.60@tcp:3:cat.0 Request processed in 201us (4560us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074756.987988:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8256 00002000:00100000:0.0:1769074756.987998:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3444736->3448831) req@ffff89b5497aad80 x1855007616656000/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/4536 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.988008:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aad80 with x1855007616656000 ext(3444736->3448831) 00002000:00010000:0.0:1769074756.988014:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aad80 00000100:00100000:0.0:1769074756.988085:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074756.988109:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616655872 00000100:00100000:0.0:1769074756.988116:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616655872 00002000:00100000:0.0:1769074756.988141:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5460ef100 x1855007616655872/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/0 e 0 to 0 dl 1769074767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074756.988150:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef100 with x1855007616655872 ext(1048576->2097151) 00002000:00010000:0.0:1769074756.988154:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5460ef100 00000100:00100000:0.0:1769074756.988170:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8257 00000100:00100000:0.0:1769074756.988177:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616655872:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074756.988974:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.989612:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074756.989651:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074756.989722:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616656128 00000100:00100000:1.0:1769074756.989731:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616656128 00000100:00100000:1.0:1769074756.989779:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616656256 00000100:00100000:1.0:1769074756.989783:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616656256 00000100:00100000:1.0:1769074756.989798:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616656384 00000100:00100000:1.0:1769074756.989800:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616656384 00000100:00100000:1.0:1769074756.989846:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13629 00000100:00100000:1.0:1769074756.989856:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616656128:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.989870:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.989889:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd5262d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aed9 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.989913:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00000100:00100000:3.0:1769074756.990091:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13630 00000100:00100000:3.0:1769074756.990100:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616656256:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074756.990108:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074756.990116:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd52634 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aee0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074756.993063:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd5262d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed9 expref: 25 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.993091:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd5262d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed9 expref: 25 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074756.993105:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866a00, rc 0) 00000100:00100000:3.0:1769074756.993311:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625051904:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074756.993412:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074756.994046:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616656128:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4188us (5072us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074756.994057:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13629 00000100:00100000:1.0:1769074756.994090:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13631 00000100:00100000:1.0:1769074756.994103:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616656384:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074756.994118:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074756.995337:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5263b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aee7 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074756.996105:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052032:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074756.996778:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00002000:00010000:3.0:1769074757.010683:0:36516:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074755 -> 1769074756 00002000:00010000:3.0:1769074757.010694:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074755 -> 1769074756 00002000:00010000:3.0:1769074757.010696:0:36516:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074755 -> 1769074756 00000100:00100000:3.0:1769074757.010721:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625051904:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074757.010759:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd52634 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aee0 expref: 23 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.010785:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd52634 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aee0 expref: 23 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.010797:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877800, rc 0) 00000100:00100000:1.0:1769074757.010962:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616655744:12345-192.168.204.60@tcp:3:cat.0 Request processed in 28596us (28839us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074757.010974:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8255 00002000:00100000:1.0:1769074757.010987:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54441c700 x1855007616655744/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.010997:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c700 with x1855007616655744 ext(1048576->2097151) 00002000:00010000:1.0:1769074757.011002:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441c700 00000100:00100000:3.0:1769074757.020298:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616655616:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37999us (38215us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074757.020313:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8254 00002000:00100000:3.0:1769074757.020326:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441c380 x1855007616655616/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.020337:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c380 with x1855007616655616 ext(1048576->2097151) 00002000:00010000:3.0:1769074757.020342:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441c380 00010000:00010000:0.0:1769074757.021325:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074757.026377:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:0.0:1769074757.026836:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074757.029734:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.030160:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616655872:12345-192.168.204.60@tcp:3:cat.0 Request processed in 41982us (42074us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074757.030170:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8257 00002000:00100000:1.0:1769074757.030181:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5460ef100 x1855007616655872/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.030191:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef100 with x1855007616655872 ext(1048576->2097151) 00002000:00010000:1.0:1769074757.030196:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5460ef100 00000100:00080000:0.0:1769074757.031730:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074757.037768:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616655488:12345-192.168.204.60@tcp:3:cat.0 Request processed in 51048us (54529us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074757.037786:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 670 00002000:00100000:1.0:1769074757.037798:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5497a9f80 x1855007616655488/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:292/0 lens 488/440 e 0 to 0 dl 1769074767 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.037850:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9f80 with x1855007616655488 ext(0->1048575) 00010000:00010000:1.0:1769074757.037864:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 11 pid: 27843 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074757.037883:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5253f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae54 expref: 11 pid: 27843 timeout: 1844 lvb_type: 1 00002000:00010000:1.0:1769074757.037890:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a9f80 00000100:00100000:0.0:1769074757.053290:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:0.0:1769074757.056892:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074757.061367:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.061431:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657024 00000100:00100000:1.0:1769074757.061442:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657024 00002000:00100000:1.0:1769074757.061496:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (905216->909311) req@ffff89b54d0e4380 x1855007616657024/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.061508:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4380 with x1855007616657024 ext(905216->909311) 00010000:00010000:1.0:1769074757.061517:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 23 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074757.061533:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 23 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074757.061541:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d0e4380 00000100:00100000:1.0:1769074757.061662:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 671 00000100:00100000:1.0:1769074757.061673:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616657024:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074757.061935:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616657024:12345-192.168.204.60@tcp:3:cat.0 Request processed in 256us (568us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074757.061943:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 671 00002000:00100000:1.0:1769074757.061952:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (905216->909311) req@ffff89b54d0e4380 x1855007616657024/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/4536 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.061961:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4380 with x1855007616657024 ext(905216->909311) 00010000:00010000:1.0:1769074757.061966:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 23 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074757.061976:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 23 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074757.061982:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d0e4380 00000100:00100000:0.0:1769074757.070353:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.070458:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.070586:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657152 00000100:00100000:3.0:1769074757.070597:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657152 00000100:00100000:3.0:1769074757.070658:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13633 00000100:00100000:3.0:1769074757.070668:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62516:x1855007616657152:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074757.070690:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.070713:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aef5 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.070748:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.070758:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.070767:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 9 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.070775:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.070778:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.070782:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->1925119] (req 1048576->1925119) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.070856:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 9 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.070869:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 9 pid: 27843 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074757.070885:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052416:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.071004:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.071014:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 11 pid: 27844 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074757.071023:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052544:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.071055:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.071069:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:3.0:1769074757.071076:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052672:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.071097:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074757.072832:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657280 00000100:00100000:1.0:1769074757.072847:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657280 00002000:00100000:1.0:1769074757.072910:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (905216->1048575) req@ffff89b546f91f80 x1855007616657280/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.073098:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91f80 with x1855007616657280 ext(905216->1048575) 00010000:00010000:1.0:1769074757.073111:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 23 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:1.0:1769074757.073126:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 23 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:1.0:1769074757.073134:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f91f80 00000100:00100000:1.0:1769074757.073234:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 672 00000100:00100000:1.0:1769074757.073246:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616657280:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074757.075952:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052416:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.076236:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052544:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.076247:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625052672:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.076293:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->1925119] (req 1048576->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.076318:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:3.0:1769074757.076380:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62516:x1855007616657152:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 5714us (6041us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.076390:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13633 00000100:00100000:0.0:1769074757.079970:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616657280:12345-192.168.204.60@tcp:4:cat.0 Request processed in 6716us (9500us total) trans 17179874460 rc 0/0 00000100:00100000:0.0:1769074757.079984:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 672 00002000:00100000:0.0:1769074757.079996:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (905216->1048575) req@ffff89b546f91f80 x1855007616657280/t17179874460(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/448 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.080006:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91f80 with x1855007616657280 ext(905216->1048575) 00010000:00010000:0.0:1769074757.080017:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 25 pid: 10284 timeout: 1844 lvb_type: 0 00010000:00010000:0.0:1769074757.080031:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd52603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 905216->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aecb expref: 25 pid: 10284 timeout: 1844 lvb_type: 0 00002000:00010000:0.0:1769074757.080039:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f91f80 00000100:00100000:1.0:1769074757.080819:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657536 00000100:00100000:1.0:1769074757.080827:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657536 00010000:00010000:1.0:1769074757.080858:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 1844 lvb_type: 1 00000100:00100000:1.0:1769074757.080929:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657408 00000100:00100000:1.0:1769074757.080933:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657408 00010000:00010000:1.0:1769074757.080947:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 12 pid: 27844 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074757.081167:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657664 00000100:00100000:0.0:1769074757.081179:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657664 00010000:00010000:0.0:1769074757.081238:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 10 pid: 27843 timeout: 1844 lvb_type: 1 00000100:00100000:0.0:1769074757.081260:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6732 00000100:00100000:0.0:1769074757.081267:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616657536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.081280:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.081297:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074757.081308:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074757.081330:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5250e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae31 expref: 18 pid: 36516 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074757.081347:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.081382:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616657536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 116us (5297us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.081392:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6732 00000100:00100000:0.0:1769074757.081450:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6733 00000100:00100000:0.0:1769074757.081456:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616657408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.081462:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1769074757.081842:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6734 00000100:00100000:1.0:1769074757.081852:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf94a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616657664:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074757.081861:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074757.081873:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 10 pid: 27843 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074757.081886:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 10 pid: 27843 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074757.081899:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd5254d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae5b expref: 10 pid: 27843 timeout: 1844 lvb_type: 1 00010000:00010000:1.0:1769074757.081909:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074757.081952:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616657664:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 103us (1196us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074757.081958:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6734 00010000:00010000:0.0:1769074757.082352:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 12 pid: 27844 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074757.082365:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 12 pid: 27844 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074757.082379:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd5257e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae7e expref: 12 pid: 27844 timeout: 1844 lvb_type: 1 00010000:00010000:0.0:1769074757.082389:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.082391:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.082398:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.082409:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.082477:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074757.082492:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625052800:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074757.082552:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.088727:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.092991:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625052800:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074757.093018:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074757.093493:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf94380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616657408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12033us (17301us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074757.093506:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6733 00000100:00100000:3.0:1769074757.097941:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.097984:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.098103:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616657920 00000100:00100000:0.0:1769074757.098115:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616657920 00000100:00100000:1.0:1769074757.098281:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616658048 00000100:00100000:1.0:1769074757.098286:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616658048 00002000:00100000:1.0:1769074757.098305:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d0e7100 x1855007616658048/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 504/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.098316:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e7100 with x1855007616658048 ext(2097152->4194303) 00002000:00010000:1.0:1769074757.098322:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e7100 00000100:00100000:1.0:1769074757.098349:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8258 00000100:00100000:1.0:1769074757.098355:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616658048:12345-192.168.204.60@tcp:3:cat.0 00002000:00100000:0.0:1769074757.099399:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5436d6680 x1855007616657920/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 504/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.099409:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6680 with x1855007616657920 ext(2097152->4194303) 00002000:00010000:0.0:1769074757.099414:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6680 00000100:00100000:0.0:1769074757.099436:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8259 00000100:00100000:0.0:1769074757.099446:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616657920:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074757.120664:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616658176 00000100:00100000:1.0:1769074757.120684:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616658176 00002000:00100000:1.0:1769074757.120874:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1921024->1925119) req@ffff89b54cf95500 x1855007616658176/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.121039:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf95500 with x1855007616658176 ext(1921024->1925119) 00002000:00010000:1.0:1769074757.121299:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf95500 00000100:00100000:1.0:1769074757.121425:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616658304 00000100:00100000:1.0:1769074757.121439:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616658304 00002000:00100000:1.0:1769074757.121465:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54cf95880 x1855007616658304/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.121475:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf95880 with x1855007616658304 ext(0->1048575) 00002000:00010000:1.0:1769074757.121479:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf95880 00000100:00100000:1.0:1769074757.121514:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8260 00000100:00100000:1.0:1769074757.121523:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616658176:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074757.121751:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616658176:12345-192.168.204.60@tcp:3:cat.0 Request processed in 228us (4100us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074757.121759:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8260 00002000:00100000:1.0:1769074757.121766:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1921024->1925119) req@ffff89b54cf95500 x1855007616658176/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/4536 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.121775:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf95500 with x1855007616658176 ext(1921024->1925119) 00002000:00010000:1.0:1769074757.121779:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf95500 00000100:00100000:1.0:1769074757.121830:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8261 00000100:00100000:1.0:1769074757.121837:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616658304:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074757.129465:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616657920:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30010us (31523us total) trans 0 rc 1794048/1794048 00000100:00100000:0.0:1769074757.129481:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8259 00002000:00100000:0.0:1769074757.129495:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5436d6680 x1855007616657920/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 504/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1794048/1794048 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.129507:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6680 with x1855007616657920 ext(2097152->4194303) 00002000:00010000:0.0:1769074757.129514:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6680 00000100:00100000:1.0:1769074757.142747:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616658048:12345-192.168.204.60@tcp:3:cat.0 Request processed in 44379us (44751us total) trans 0 rc 1736704/1736704 00000100:00100000:1.0:1769074757.142760:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8258 00002000:00100000:1.0:1769074757.142772:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d0e7100 x1855007616658048/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 504/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1736704/1736704 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.142782:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e7100 with x1855007616658048 ext(2097152->4194303) 00002000:00010000:1.0:1769074757.142788:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e7100 00000100:00100000:0.0:1769074757.177500:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.177616:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.177656:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.186378:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616658560 00000100:00100000:3.0:1769074757.186553:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616658560 00000100:00080000:1.0:1769074757.187283:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074757.190460:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.200139:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.200204:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.200232:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616659456 00000100:00100000:0.0:1769074757.200242:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616659456 00000100:00100000:0.0:1769074757.200291:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13225 00000100:00100000:0.0:1769074757.200301:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62532:x1855007616659456:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074757.200324:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074757.200345:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5265e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af0a expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200478:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.200495:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200503:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 16 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200510:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 226 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200519:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 6/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.200556:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 226 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200580:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 226 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.200594:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625053184:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074757.200684:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 16 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200693:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 16 pid: 6512 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.200706:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625053312:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074757.200738:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.200747:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 13 pid: 6512 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.200755:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625053440:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.200777:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074757.200865:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.200895:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616659072 00000100:00100000:0.0:1769074757.200901:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616659072 00002000:00100000:0.0:1769074757.200944:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546f90e00 x1855007616659072/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.200954:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90e00 with x1855007616659072 ext(1048576->2097151) 00002000:00010000:0.0:1769074757.200960:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f90e00 00000100:00100000:0.0:1769074757.200976:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616659200 00000100:00100000:0.0:1769074757.200979:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616659200 00002000:00100000:0.0:1769074757.200996:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546f92680 x1855007616659200/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.201005:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92680 with x1855007616659200 ext(1048576->2097151) 00002000:00010000:0.0:1769074757.201008:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f92680 00000100:00100000:0.0:1769074757.201023:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8263 00000100:00100000:0.0:1769074757.201030:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616659072:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074757.201617:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8264 00000100:00100000:0.0:1769074757.201625:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616659200:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074757.207150:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616658944:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16479us (16698us total) trans 0 rc 172032/172032 00000100:00100000:1.0:1769074757.207166:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8262 00002000:00100000:1.0:1769074757.207179:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1925120->2097151) req@ffff89b5446c0700 x1855007616658944/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 172032/172032 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.207189:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0700 with x1855007616658944 ext(1925120->2097151) 00002000:00010000:1.0:1769074757.207195:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c0700 00000100:00100000:3.0:1769074757.208579:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625053184:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.208597:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625053312:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.216211:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616659712 00000100:00100000:0.0:1769074757.216220:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616659712 00010000:00010000:0.0:1769074757.216257:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 17 pid: 6512 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.220279:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6736 00000100:00100000:0.0:1769074757.220303:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616659712:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.220328:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.220341:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 17 pid: 6512 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.220359:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 17 pid: 6512 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.220389:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd525ee lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aebd expref: 17 pid: 6512 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.220407:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.220412:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.220424:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.220431:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.220506:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616659712:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 216us (5465us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.220515:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6736 00000100:00100000:0.0:1769074757.221749:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.221786:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616659584 00000100:00100000:0.0:1769074757.221795:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616659584 00010000:00010000:0.0:1769074757.221841:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 227 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.221871:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6737 00000100:00100000:0.0:1769074757.221879:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2407:x1855007616659584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.221892:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.221899:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 227 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.221925:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 227 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.221946:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd5261f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aed2 expref: 227 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.221960:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.221965:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.221973:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.221979:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.222015:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2407:x1855007616659584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 137us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.222023:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6737 00000100:00100000:0.0:1769074757.222497:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.222517:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616659968 00000100:00100000:0.0:1769074757.222522:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616659968 00010000:00010000:0.0:1769074757.222555:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 14 pid: 6512 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.222592:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6738 00000100:00100000:0.0:1769074757.222600:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616659968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.222609:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.222614:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 14 pid: 6512 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.222623:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 14 pid: 6512 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.222637:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.222640:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.222649:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.222656:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.222673:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.222694:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616659968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 95us (199us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.222704:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6738 00000100:00100000:0.0:1769074757.222971:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616659072:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21940us (22845us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074757.222980:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8263 00002000:00100000:0.0:1769074757.222989:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546f90e00 x1855007616659072/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.223000:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90e00 with x1855007616659072 ext(1048576->2097151) 00002000:00010000:0.0:1769074757.223004:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f90e00 00010000:00010000:3.0:1769074757.223835:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd525bd lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aea8 expref: 13 pid: 6512 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.223856:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625053440:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.223887:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.223896:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.223903:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074757.223926:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074757.224059:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5265e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 17 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.224073:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.224080:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074757.224083:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1769074757.224086:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877e00, rc 0) 00000100:00100000:3.0:1769074757.224132:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62532:x1855007616659456:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 23833us (23927us total) trans 8589943815 rc 0/0 00000100:00100000:3.0:1769074757.224140:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13225 00000100:00080000:1.0:1769074757.230266:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074757.233590:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616659200:12345-192.168.204.60@tcp:3:cat.0 Request processed in 31939us (32698us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074757.233606:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8264 00002000:00100000:0.0:1769074757.233619:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546f92680 x1855007616659200/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.233630:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92680 with x1855007616659200 ext(1048576->2097151) 00002000:00010000:0.0:1769074757.233636:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f92680 00000100:00100000:1.0:1769074757.235359:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616659840:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20092us (20251us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074757.235374:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8265 00002000:00100000:1.0:1769074757.235399:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5475db100 x1855007616659840/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.235410:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475db100 with x1855007616659840 ext(4194304->4808703) 00002000:00010000:1.0:1769074757.235418:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475db100 00000100:00100000:1.0:1769074757.261510:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.261562:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.261623:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.261705:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616660224 00000100:00100000:3.0:1769074757.261709:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616660224 00000100:00100000:3.0:1769074757.264955:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13637 00000100:00100000:3.0:1769074757.264978:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616660352:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074757.264998:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.265020:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5267a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af1f expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074757.265084:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625053696:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074757.265183:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.265211:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13638 00000100:00100000:3.0:1769074757.265220:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616660224:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074757.265230:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.265239:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd52681 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af18 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074757.265269:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625053824:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074757.265295:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.282918:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.283622:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625053696:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074757.283659:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5267a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af1f expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074757.283681:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5267a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af1f expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074757.283694:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:1.0:1769074757.283741:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616660352:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18773us (22177us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074757.283748:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13637 00000100:00100000:0.0:1769074757.284197:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625053824:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074757.284218:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd52681 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af18 expref: 25 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074757.284231:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd52681 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af18 expref: 25 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074757.284241:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:0.0:1769074757.284284:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616660224:12345-192.168.204.60@tcp:101:cat.0 Request processed in 19065us (22660us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074757.284291:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13638 00000100:00100000:3.0:1769074757.294648:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625053952:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074757.294687:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52688 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af26 expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.294711:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd52688 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af26 expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.294724:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3c00, rc 0) 00000100:00100000:3.0:1769074757.294790:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616660480:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11618us (11889us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074757.294802:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13639 00000100:00100000:3.0:1769074757.303682:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.313331:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.313544:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616660864 00000100:00100000:0.0:1769074757.313554:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616660864 00000100:00100000:0.0:1769074757.313614:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4604 00000100:00100000:0.0:1769074757.313623:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62526:x1855007616660864:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:0.0:1769074757.313882:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62526:x1855007616660864:12345-192.168.204.60@tcp:35:cat.0 Request processed in 260us (560us total) trans 8589943816 rc 0/0 00000100:00100000:0.0:1769074757.313890:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4604 00000100:00100000:0.0:1769074757.314591:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13640 00000100:00100000:0.0:1769074757.314599:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62529:x1855007616660608:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074757.314615:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074757.314634:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af2d expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.314667:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd525fc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aec4 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.314676:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074757.314678:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074757.314683:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 25 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.314718:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd525fc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aec4 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.314735:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd525fc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aec4 expref: 10 pid: 27843 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.314749:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625054080:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.314896:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074757.331049:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625054080:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074757.331085:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 25 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.331105:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:0.0:1769074757.331170:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62529:x1855007616660608:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16570us (27501us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.331180:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13640 00000100:00100000:3.0:1769074757.331513:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.348872:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616660992:12345-192.168.204.60@tcp:4:cat.0 Request processed in 17079us (17347us total) trans 17179874461 rc 0/0 00000100:00100000:3.0:1769074757.348888:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 673 00002000:00100000:3.0:1769074757.348902:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3448831) req@ffff89b546f94380 x1855007616660992/t17179874461(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/448 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.350224:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94380 with x1855007616660992 ext(3145728->3448831) 00010000:00010000:3.0:1769074757.350243:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd525fc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aec4 expref: 13 pid: 27843 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.350266:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd525fc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aec4 expref: 13 pid: 27843 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.350274:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f94380 00000100:00100000:3.0:1769074757.374383:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.374436:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.374664:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616661248 00000100:00100000:0.0:1769074757.374677:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616661248 00000100:00100000:0.0:1769074757.374736:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13226 00000100:00100000:0.0:1769074757.374748:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:62535:x1855007616661248:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074757.374782:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074757.374834:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd52696 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af34 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.374928:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.374945:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.374958:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.374971:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.374977:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.374982:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375108:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375120:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375127:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:0.0:1769074757.375137:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074757.375228:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375235:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375241:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375247:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375254:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b053000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074757.375258:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375265:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375278:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074757.375281:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074757.375286:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375292:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053000/0x7fe2323adfd5269d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375307:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375312:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375318:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375323:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b053c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074757.375326:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375332:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375339:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47a00) 00010000:00010000:0.0:1769074757.375341:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074757.375344:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375352:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053c00/0x7fe2323adfd526a4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.375394:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd52696 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 229 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.375408:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 228 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.375415:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.375418:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074757.375420:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052000, rc 0) 00000100:00100000:0.0:1769074757.375479:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:62535:x1855007616661248:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 734us (1108us total) trans 8589943817 rc 0/0 00000100:00100000:0.0:1769074757.375487:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13226 00000100:00100000:0.0:1769074757.386293:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625054208:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074757.386334:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.386413:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616661376:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9405us (11974us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.386425:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6739 00000100:00100000:1.0:1769074757.412255:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.412364:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616661504 00000100:00100000:3.0:1769074757.412376:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616661504 00002000:00100000:3.0:1769074757.412432:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5446c0380 x1855007616661504/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.412444:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0380 with x1855007616661504 ext(2097152->3145727) 00002000:00010000:3.0:1769074757.412451:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c0380 00000100:00100000:3.0:1769074757.412491:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8266 00000100:00100000:3.0:1769074757.412501:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616661504:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074757.419247:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.426518:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616661504:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14007us (14264us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074757.426533:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8266 00002000:00100000:3.0:1769074757.426547:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5446c0380 x1855007616661504/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.426558:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0380 with x1855007616661504 ext(2097152->3145727) 00002000:00010000:3.0:1769074757.426576:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c0380 00000100:00100000:0.0:1769074757.427508:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616661632 00000100:00100000:0.0:1769074757.427519:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616661632 00000100:00100000:1.0:1769074757.428608:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13641 00000100:00100000:1.0:1769074757.428620:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62532:x1855007616661632:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074757.428639:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074757.428658:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af3b expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.428691:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 24 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.428700:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074757.428702:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074757.428706:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2568192->3145727] (req 2568192->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 13 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.428745:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 24 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.428758:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 24 pid: 10284 timeout: 1845 lvb_type: 0 00000100:00100000:1.0:1769074757.428772:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625054336:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074757.431800:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.440789:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616661760 00000100:00100000:3.0:1769074757.440799:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616661760 00000100:00100000:3.0:1769074757.440869:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13642 00000100:00100000:3.0:1769074757.440879:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62529:x1855007616661760:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074757.440897:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.441208:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af42 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.441239:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.441248:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074757.441251:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074757.441255:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.441292:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.441306:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 8 pid: 27843 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.441320:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625054464:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.441405:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.442549:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625054336:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.442592:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2568192->3145727] (req 2568192->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 13 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.442607:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640031200, rc 0) 00000100:00100000:3.0:1769074757.442666:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62532:x1855007616661632:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 14047us (23431us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.442674:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13641 00000100:00100000:1.0:1769074757.458196:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.459362:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625054464:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.459406:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.459425:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:3.0:1769074757.459479:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62529:x1855007616661760:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18601us (19273us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.459490:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13642 00000100:00100000:0.0:1769074757.462515:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616661888 00000100:00100000:0.0:1769074757.462528:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616661888 00010000:00010000:1.0:1769074757.462645:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 25 pid: 10284 timeout: 1845 lvb_type: 0 00000100:00100000:1.0:1769074757.462686:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6740 00000100:00100000:1.0:1769074757.462705:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616661888:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074757.462722:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074757.462745:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 25 pid: 10284 timeout: 1845 lvb_type: 0 00010000:00010000:1.0:1769074757.462759:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 25 pid: 10284 timeout: 1845 lvb_type: 0 00010000:00010000:1.0:1769074757.462784:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5268f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af2d expref: 25 pid: 10284 timeout: 1845 lvb_type: 0 00010000:00010000:1.0:1769074757.462801:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074757.462803:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074757.462852:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.462863:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.462902:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074757.462927:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625054592:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074757.463010:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a3970c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074757.495415:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616662144 00000100:00100000:1.0:1769074757.495426:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616662144 00002000:00100000:1.0:1769074757.495481:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2568192->2572287) req@ffff89b542fa8000 x1855007616662144/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.495498:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8000 with x1855007616662144 ext(2568192->2572287) 00002000:00010000:1.0:1769074757.495504:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa8000 00000100:00100000:1.0:1769074757.495540:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8267 00000100:00100000:1.0:1769074757.495549:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616662144:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074757.495918:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616662144:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 358us (621us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074757.495927:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8267 00002000:00100000:1.0:1769074757.495934:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2568192->2572287) req@ffff89b542fa8000 x1855007616662144/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/4536 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074757.495942:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8000 with x1855007616662144 ext(2568192->2572287) 00002000:00010000:1.0:1769074757.495945:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa8000 00000100:00100000:1.0:1769074757.525626:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616662272 00000100:00100000:1.0:1769074757.525637:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616662272 00010000:00010000:1.0:1769074757.525677:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 11 pid: 27843 timeout: 1845 lvb_type: 0 00000100:00100000:1.0:1769074757.525732:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6741 00000100:00100000:1.0:1769074757.525741:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616662272:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074757.525761:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074757.525785:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 11 pid: 27843 timeout: 1845 lvb_type: 0 00010000:00010000:1.0:1769074757.525800:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 11 pid: 27843 timeout: 1845 lvb_type: 0 00010000:00010000:1.0:1769074757.525875:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd525d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeb6 expref: 11 pid: 27843 timeout: 1845 lvb_type: 0 00010000:00010000:1.0:1769074757.525891:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074757.525893:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074757.525900:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.526197:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.526243:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074757.526258:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625054720:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074757.526357:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074757.533374:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.534202:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616662400 00000100:00100000:3.0:1769074757.534215:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616662400 00000100:00100000:3.0:1769074757.534283:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13643 00000100:00100000:3.0:1769074757.534295:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62532:x1855007616662400:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074757.534322:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.534347:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af49 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.534382:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.534392:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.534394:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.534398:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->2265087] (req 2097152->2265087) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.534438:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.534451:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 17 pid: 8398 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.534466:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625054848:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.539763:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074757.542676:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625054720:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074757.542721:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074757.542868:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616662272:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17125us (17576us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074757.542884:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6741 00000100:00100000:3.0:1769074757.550107:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.550529:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616662528 00000100:00100000:0.0:1769074757.550540:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616662528 00002000:00100000:0.0:1769074757.550603:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b546f96680 x1855007616662528/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.550615:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96680 with x1855007616662528 ext(2097152->3145727) 00010000:00010000:0.0:1769074757.550624:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 18 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.550641:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 18 pid: 8398 timeout: 1845 lvb_type: 0 00002000:00010000:0.0:1769074757.550649:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f96680 00000100:00100000:0.0:1769074757.550695:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 675 00000100:00100000:0.0:1769074757.550706:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616662528:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074757.600712:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616662528:12345-192.168.204.60@tcp:3:cat.0 Request processed in 49985us (50593us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074757.600730:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 675 00002000:00100000:0.0:1769074757.600748:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b546f96680 x1855007616662528/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.600761:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96680 with x1855007616662528 ext(2097152->3145727) 00010000:00010000:0.0:1769074757.600774:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 18 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.600795:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd526c0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af42 expref: 18 pid: 8398 timeout: 1845 lvb_type: 0 00002000:00010000:0.0:1769074757.600804:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f96680 00000100:00100000:0.0:1769074757.624793:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.625254:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616662784 00000100:00100000:3.0:1769074757.625266:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616662784 00000100:00100000:3.0:1769074757.625326:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13227 00000100:00100000:3.0:1769074757.625337:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62539:x1855007616662784:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074757.625362:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.625386:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd526ce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af50 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.625459:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625473:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625483:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625498:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625505:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625511:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625636:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625647:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625654:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074757.625668:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074757.625743:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625751:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625758:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625764:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625771:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074757.625775:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625784:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625797:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074757.625800:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074757.625831:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625838:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd526d5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625851:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625856:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625865:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625869:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b437400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074757.625872:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625877:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625884:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ea900) 00010000:00010000:3.0:1769074757.625887:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074757.625890:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074757.625896:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd526dc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074757.626144:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:3.0:1769074757.626239:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd526ce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 20 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.626254:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 19 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.626261:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074757.626264:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074757.626266:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437e00, rc 0) 00000100:00100000:3.0:1769074757.626675:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62539:x1855007616662784:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1340us (1896us total) trans 8589943818 rc 0/0 00000100:00100000:3.0:1769074757.626684:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13227 00000100:00100000:1.0:1769074757.644568:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.683287:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.683334:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.683471:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616663168 00000100:00100000:1.0:1769074757.683482:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616663168 00000100:00100000:1.0:1769074757.683535:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616663296 00000100:00100000:1.0:1769074757.683538:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616663296 00000100:00100000:1.0:1769074757.683556:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13645 00000100:00100000:1.0:1769074757.683575:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616663168:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074757.683594:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074757.683613:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd526f8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af5e expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074757.683666:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625055232:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074757.683848:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074757.685397:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.685858:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616663424 00000100:00100000:3.0:1769074757.685869:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616663424 00002000:00100000:3.0:1769074757.686144:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2568192->3145727) req@ffff89b549f28a80 x1855007616663424/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.686155:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28a80 with x1855007616663424 ext(2568192->3145727) 00010000:00010000:3.0:1769074757.686164:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 13 pid: 10284 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.686181:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 13 pid: 10284 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.686190:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f28a80 00000100:00100000:3.0:1769074757.686227:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 676 00000100:00100000:3.0:1769074757.686238:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616663424:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074757.697964:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:3.0:1769074757.704451:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074756 -> 1769074757 00002000:00010000:3.0:1769074757.704461:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074756 -> 1769074757 00000100:00100000:3.0:1769074757.704485:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625055360:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074757.704522:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd526ff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af65 expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.704543:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd526ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af65 expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.704557:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924a00, rc 0) 00000100:00100000:3.0:1769074757.704626:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616663296:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20372us (21291us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074757.704639:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13646 00000100:00100000:3.0:1769074757.713339:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616663424:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 27087us (27933us total) trans 17179874462 rc 0/0 00000100:00100000:3.0:1769074757.713355:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 676 00002000:00100000:3.0:1769074757.713369:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2568192->3145727) req@ffff89b549f28a80 x1855007616663424/t17179874462(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/448 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.713380:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28a80 with x1855007616663424 ext(2568192->3145727) 00010000:00010000:3.0:1769074757.713390:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 15 pid: 10284 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.713408:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd526b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2568192->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af3b expref: 15 pid: 10284 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.713416:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f28a80 00000100:00100000:0.0:1769074757.720929:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.730476:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.730686:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616663808 00000100:00100000:3.0:1769074757.730703:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616663808 00002000:00100000:3.0:1769074757.730749:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54538b100 x1855007616663808/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.730759:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54538b100 with x1855007616663808 ext(3145728->4194303) 00002000:00010000:3.0:1769074757.730764:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54538b100 00000100:00100000:3.0:1769074757.730798:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8269 00000100:00100000:3.0:1769074757.730837:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616663808:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074757.752997:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.753749:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.754936:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664064 00000100:00100000:3.0:1769074757.754946:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664064 00000100:00100000:3.0:1769074757.754998:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13228 00000100:00100000:3.0:1769074757.755006:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62540:x1855007616664064:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.755025:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.755051:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52706 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af6c expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.755114:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755129:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755139:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755153:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755159:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755165:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755236:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755249:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 18 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.755256:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.755290:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 18 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.755302:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 18 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.755315:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625055616:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.756269:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.773486:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664192 00000100:00100000:3.0:1769074757.773497:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664192 00000100:00100000:3.0:1769074757.773556:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13647 00000100:00100000:3.0:1769074757.773576:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62529:x1855007616664192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074757.773598:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.773621:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af73 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.773654:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.773677:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074757.773679:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074757.773684:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.773725:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.773739:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 10 pid: 8398 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.773762:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625055744:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.773871:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656049c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.773935:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625055616:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.773955:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000100:00100000:1.0:1769074757.782791:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664448 00000100:00100000:1.0:1769074757.782825:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664448 00000100:00100000:1.0:1769074757.782902:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4605 00000100:00100000:1.0:1769074757.782914:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:57730:x1855007616664448:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:1.0:1769074757.783234:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:57730:x1855007616664448:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 321us (10278us total) trans 8589943819 rc 0/0 00000100:00100000:1.0:1769074757.783242:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4605 00000100:00100000:3.0:1769074757.783713:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625055744:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.783759:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.783785:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00000100:00100000:3.0:1769074757.783857:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62529:x1855007616664192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10283us (10456us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.783865:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13647 00000100:00100000:3.0:1769074757.791715:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664576 00000100:00100000:3.0:1769074757.791728:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664576 00002000:00100000:3.0:1769074757.791781:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2265087) req@ffff89b547ef7800 x1855007616664576/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.791794:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7800 with x1855007616664576 ext(2097152->2265087) 00010000:00010000:3.0:1769074757.791804:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 11 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.798660:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 11 pid: 8398 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.798679:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef7800 00000100:00100000:3.0:1769074757.798773:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 677 00000100:00100000:3.0:1769074757.798789:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616664576:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074757.803464:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616663936:12345-192.168.204.60@tcp:3:cat.0 Request processed in 49521us (50481us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074757.803478:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8270 00002000:00100000:0.0:1769074757.803493:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b546f93480 x1855007616663936/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074757.803505:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93480 with x1855007616663936 ext(0->1048575) 00002000:00010000:0.0:1769074757.803511:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f93480 00000100:00100000:3.0:1769074757.803765:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664704 00000100:00100000:3.0:1769074757.803777:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664704 00010000:00010000:3.0:1769074757.803844:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 20 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.803872:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6744 00000100:00100000:3.0:1769074757.803880:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616664704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074757.803896:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074757.805318:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 20 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.805344:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 20 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.805365:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd52665 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af0a expref: 20 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.805845:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074757.805854:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.805865:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.805877:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.807398:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074757.807490:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616664704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3602us (4996us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.807503:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6744 00000100:00100000:3.0:1769074757.810960:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616664576:12345-192.168.204.60@tcp:4:cat.0 Request processed in 12166us (19417us total) trans 21474841137 rc 0/0 00000100:00100000:3.0:1769074757.810978:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 677 00002000:00100000:3.0:1769074757.810995:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2265087) req@ffff89b547ef7800 x1855007616664576/t21474841137(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/448 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.811297:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7800 with x1855007616664576 ext(2097152->2265087) 00010000:00010000:3.0:1769074757.811309:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.811327:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.811336:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef7800 00010000:00010000:0.0:1769074757.814783:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814802:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:0.0:1769074757.814891:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:0.0:1769074757.814896:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074757.814933:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814949:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814961:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814968:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814974:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814981:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814986:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b437600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074757.814990:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.814996:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815022:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074757.815025:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074757.815029:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815036:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd5270d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815062:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815068:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815074:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815079:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074757.815082:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815092:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815099:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ea900) 00010000:00010000:0.0:1769074757.815101:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074757.815105:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815110:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52714 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.815152:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd52706 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 15 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.815166:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.815172:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.815175:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074757.815177:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45a00, rc 0) 00000100:00100000:0.0:1769074757.815248:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62540:x1855007616664064:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 60242us (61498us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.815257:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13228 00000100:00100000:3.0:1769074757.824191:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664832 00000100:00100000:3.0:1769074757.824202:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664832 00000100:00100000:3.0:1769074757.824253:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616664960 00000100:00100000:3.0:1769074757.824257:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616664960 00000100:00100000:3.0:1769074757.824277:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665088 00000100:00100000:3.0:1769074757.824280:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665088 00000100:00100000:3.0:1769074757.824294:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665216 00000100:00100000:3.0:1769074757.824298:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665216 00000100:00100000:3.0:1769074757.824315:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13648 00000100:00100000:3.0:1769074757.824324:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616664832:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.824341:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.824361:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af7a expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.824385:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.824388:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.824399:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.824415:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.824426:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45200, rc 0) 00000100:00100000:3.0:1769074757.824478:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2406:x1855007616664832:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 155us (459us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.824485:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13648 00000100:00100000:3.0:1769074757.824503:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13649 00000100:00100000:3.0:1769074757.824510:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616664960:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.824517:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.824527:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52730 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af81 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074757.824583:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625055872:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074757.824635:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656049c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.824657:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13650 00000100:00100000:3.0:1769074757.824665:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616665088:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.824672:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.824680:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af88 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.824692:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.824693:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.824699:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.824709:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.824718:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45600, rc 0) 00000100:00100000:3.0:1769074757.824739:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2406:x1855007616665088:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 75us (642us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.824746:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13650 00000100:00100000:3.0:1769074757.824761:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13651 00000100:00100000:3.0:1769074757.824768:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616665216:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.824774:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.824784:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5273e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af8f expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074757.824842:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625056000:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074757.824870:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.831505:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665344 00000100:00100000:3.0:1769074757.831518:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665344 00010000:00010000:3.0:1769074757.831575:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.831675:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6745 00000100:00100000:3.0:1769074757.831685:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616665344:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074757.831704:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074757.831725:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.831739:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.831763:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.831765:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.831773:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.831784:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.831869:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074757.831883:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625056128:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074757.831970:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00010000:00010000:3.0:1769074757.833849:0:27843:0:(ldlm_lockd.c:740:ldlm_handle_ast_error()) ### client (nid 192.168.204.60@tcp) returned -22 from glimpse AST (req@00000000e1ba0e50 x1855007625056000) - normal race ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.833892:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd526c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2265087) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af49 expref: 13 pid: 8398 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.833908:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625056000:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074757.833942:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5273e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af8f expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.833958:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5273e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af8f expref: 14 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.833967:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba400, rc 0) 00000100:00100000:3.0:1769074757.834037:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616665216:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9268us (9916us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074757.834051:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13651 00002000:00010000:3.0:1769074757.834092:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074756 -> 1769074757 00002000:00010000:3.0:1769074757.834095:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074756 -> 1769074757 00000100:00100000:3.0:1769074757.834103:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625055872:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074757.834118:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52730 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af81 expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.834129:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52730 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af81 expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.834142:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44000, rc 0) 00000100:00100000:3.0:1769074757.834159:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616664960:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9650us (10087us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074757.834166:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13649 00000100:00100000:1.0:1769074757.836774:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.837761:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665472 00000100:00100000:3.0:1769074757.837776:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665472 00000100:00100000:3.0:1769074757.837854:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13652 00000100:00100000:3.0:1769074757.837865:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62535:x1855007616665472:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074757.837897:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.837922:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af96 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.837957:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 19 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.837967:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.837975:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.837982:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.837984:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.837988:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1048576->1728511] (req 1048576->1728511) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.838298:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.838313:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 12 pid: 27843 timeout: 1845 lvb_type: 1 00000100:00100000:3.0:1769074757.838327:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625056256:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.838415:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.838425:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 10 pid: 27844 timeout: 1845 lvb_type: 1 00000100:00100000:3.0:1769074757.838434:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625056384:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074757.838463:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 19 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.838472:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 19 pid: 36516 timeout: 1845 lvb_type: 1 00000100:00100000:3.0:1769074757.838481:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625056512:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.838500:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656049c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074757.838963:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665728 00000100:00100000:0.0:1769074757.838969:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665728 00000100:00100000:0.0:1769074757.839018:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13229 00000100:00100000:0.0:1769074757.839026:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62540:x1855007616665728:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074757.839039:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074757.839068:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5274c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879af9d expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.839179:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.839194:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 15 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.839202:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.839223:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 15 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.839231:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 15 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.839241:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625056640:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.839278:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.840001:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625056128:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074757.840021:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074757.840065:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616665344:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8380us (9028us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.840074:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6745 00000100:00100000:1.0:1769074757.845468:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665856 00000100:00100000:1.0:1769074757.845476:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665856 00000100:00100000:1.0:1769074757.845542:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13653 00000100:00100000:1.0:1769074757.845553:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62539:x1855007616665856:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074757.845580:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074757.845596:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afa4 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.845625:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.845634:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074757.845636:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074757.845640:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 13 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.845685:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.845698:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 19 pid: 9048 timeout: 1845 lvb_type: 0 00000100:00100000:1.0:1769074757.845710:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625056768:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074757.845784:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.846470:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616665984 00000100:00100000:3.0:1769074757.846479:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616665984 00002000:00100000:3.0:1769074757.846514:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67a270700 x1855007616665984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.846525:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a270700 with x1855007616665984 ext(3145728->4194303) 00002000:00010000:3.0:1769074757.846529:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a270700 00000100:00100000:3.0:1769074757.846566:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8271 00000100:00100000:3.0:1769074757.846575:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616665984:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074757.849032:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625056256:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.849053:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625056384:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074757.849064:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625056512:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074757.849102:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1048576->1728511] (req 1048576->1728511) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.849123:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:0.0:1769074757.849189:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62535:x1855007616665472:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11326us (12426us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.849198:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13652 00000100:00100000:1.0:1769074757.852583:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.852625:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.852746:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.852914:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666368 00000100:00100000:0.0:1769074757.852928:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666368 00010000:00010000:0.0:1769074757.852974:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 16 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.853016:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666112 00000100:00100000:0.0:1769074757.853020:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666112 00010000:00010000:0.0:1769074757.853037:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 14 pid: 27843 timeout: 1845 lvb_type: 1 00000100:00100000:0.0:1769074757.853066:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666240 00000100:00100000:0.0:1769074757.853069:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666240 00010000:00010000:0.0:1769074757.853082:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 11 pid: 27844 timeout: 1845 lvb_type: 1 00000100:00100000:0.0:1769074757.853103:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6746 00000100:00100000:0.0:1769074757.853114:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616666368:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.853130:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.853137:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 16 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.853148:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 16 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.853174:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5271b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af6c expref: 16 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.853199:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.853204:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.853216:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.853223:0:6499:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.853251:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.853305:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616666368:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 193us (732us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.853314:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6746 00000100:00100000:0.0:1769074757.853367:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6747 00000100:00100000:0.0:1769074757.853374:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616666112:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.853384:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.853408:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 14 pid: 27843 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.853419:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 14 pid: 27843 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.853433:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52546 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae62 expref: 14 pid: 27843 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.853448:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.853470:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616666112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 96us (845us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.853477:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6747 00000100:00100000:0.0:1769074757.853490:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6748 00000100:00100000:0.0:1769074757.853495:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616666240:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.853503:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.853511:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 11 pid: 27844 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.853519:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 11 pid: 27844 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.853533:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52585 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae85 expref: 11 pid: 27844 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.853543:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.854443:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616666240:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 944us (1693us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.854452:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6748 00000100:00100000:0.0:1769074757.856532:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666496 00000100:00100000:0.0:1769074757.856541:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666496 00010000:00010000:0.0:1769074757.856590:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 21 pid: 36516 timeout: 1845 lvb_type: 1 00000100:00100000:0.0:1769074757.856677:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6749 00000100:00100000:0.0:1769074757.856688:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616666496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.856708:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.856733:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 21 pid: 36516 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.856749:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 21 pid: 36516 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.856778:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52500 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae38 expref: 21 pid: 36516 timeout: 1845 lvb_type: 1 00010000:00010000:0.0:1769074757.856796:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.856798:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.856836:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.856847:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.856895:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074757.856910:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625056896:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074757.856999:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074757.881922:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.882138:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616665984:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 35560us (36910us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074757.882151:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8271 00002000:00100000:3.0:1769074757.882162:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67a270700 x1855007616665984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.882171:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a270700 with x1855007616665984 ext(3145728->4194303) 00002000:00010000:3.0:1769074757.882177:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a270700 00000100:00100000:3.0:1769074757.882203:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666624 00000100:00100000:3.0:1769074757.882211:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666624 00002000:00100000:3.0:1769074757.882244:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546f90380 x1855007616666624/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.882251:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90380 with x1855007616666624 ext(1048576->2097151) 00010000:00010000:3.0:1769074757.882270:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 19 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.882287:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 19 pid: 9048 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.882294:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f90380 00000100:00100000:3.0:1769074757.882317:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 678 00000100:00100000:3.0:1769074757.882326:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616666624:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074757.883980:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625056896:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074757.884002:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.884053:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616666496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 27366us (27588us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.884063:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6749 00000100:00100000:1.0:1769074757.885629:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625056768:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.885629:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666752 00000100:00100000:3.0:1769074757.885638:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666752 00000100:00100000:3.0:1769074757.885695:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13230 00000100:00100000:3.0:1769074757.885704:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62540:x1855007616666752:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.885727:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.885746:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52768 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afab expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.885867:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.885884:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.885893:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.885927:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.885943:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 16 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.885955:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625057024:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074757.886027:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604000 going to sleep for 11 seconds 00010000:00010000:1.0:1769074757.886264:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.886284:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640030a00, rc 0) 00000100:00100000:1.0:1769074757.886316:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62539:x1855007616665856:12345-192.168.204.60@tcp:101:cat.0 Request processed in 40765us (41154us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074757.886323:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13653 00000100:00100000:3.0:1769074757.900000:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616666624:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 17661us (18075us total) trans 21474841138 rc 0/0 00000100:00100000:3.0:1769074757.900016:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 678 00002000:00100000:3.0:1769074757.900030:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546f90380 x1855007616666624/t21474841138(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/448 e 0 to 0 dl 1769074768 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.900041:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90380 with x1855007616666624 ext(1048576->2097151) 00000100:00100000:1.0:1769074757.900569:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625057024:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074757.900597:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074757.901344:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 20 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.901362:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 20 pid: 9048 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074757.901371:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f90380 00000100:00100000:1.0:1769074757.910208:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667008 00000100:00100000:1.0:1769074757.910219:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667008 00000100:00100000:1.0:1769074757.910298:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13654 00000100:00100000:1.0:1769074757.910309:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616667008:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074757.910330:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074757.910350:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52776 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afb2 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074757.910417:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625057152:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074757.910501:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074757.914866:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616666880 00000100:00100000:0.0:1769074757.914887:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616666880 00010000:00010000:0.0:1769074757.914956:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 17 pid: 6513 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.915011:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6750 00000100:00100000:0.0:1769074757.915022:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2408:x1855007616666880:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.915055:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.915063:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 17 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.915076:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 17 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.915099:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd52753 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af9d expref: 17 pid: 6513 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.915117:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074757.915124:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.915135:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.915143:0:6499:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074757.915188:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074757.915324:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.915333:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000100:00100000:0.0:1769074757.915392:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616666880:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 237us (5366us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.915404:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6750 00000004:00200000:1.0:1769074757.915412:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074757.915417:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074757.915441:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.915477:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52768 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afab expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.915493:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afab expref: 15 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.915500:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074757.915503:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074757.915506:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:1.0:1769074757.915553:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62540:x1855007616666752:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 29849us (31750us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074757.915571:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13230 00000100:00100000:3.0:1769074757.916902:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667136 00000100:00100000:3.0:1769074757.916918:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667136 00002000:00100000:3.0:1769074757.916959:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1724416->1728511) req@ffff89b54441f480 x1855007616667136/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.916968:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f480 with x1855007616667136 ext(1724416->1728511) 00002000:00010000:3.0:1769074757.916973:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441f480 00000100:00100000:3.0:1769074757.916998:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8272 00000100:00100000:3.0:1769074757.917006:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616667136:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074757.917535:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616667136:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 526us (694us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074757.917544:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8272 00002000:00100000:3.0:1769074757.917554:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1724416->1728511) req@ffff89b54441f480 x1855007616667136/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/4536 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.917573:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f480 with x1855007616667136 ext(1724416->1728511) 00002000:00010000:3.0:1769074757.917577:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441f480 00000100:00100000:0.0:1769074757.920617:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074757.924454:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667520 00000100:00100000:0.0:1769074757.924465:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667520 00010000:00010000:0.0:1769074757.924508:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074757.924545:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6751 00000100:00100000:0.0:1769074757.924567:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616667520:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074757.924582:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074757.924604:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.924620:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074757.924645:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.924647:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.924654:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.924665:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.924710:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074757.924725:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625057280:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074757.924767:0:27843:0:(ldlm_lockd.c:740:ldlm_handle_ast_error()) ### client (nid 192.168.204.60@tcp) returned -22 from glimpse AST (req@000000007e00ea49 x1855007625057152) - normal race ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074757.924790:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52649 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1925119) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aef5 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074757.924828:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625057152:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074757.924855:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52776 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb2 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.924868:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52776 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb2 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.924876:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:1.0:1769074757.928692:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667392 00000100:00100000:1.0:1769074757.928713:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667392 00000100:00100000:1.0:1769074757.931104:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13231 00000100:00100000:1.0:1769074757.931135:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62543:x1855007616667392:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074757.931182:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074757.931208:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5277d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afb9 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.931285:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931299:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931311:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931325:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931335:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931340:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931449:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931458:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931465:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074757.931474:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074757.931566:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931574:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931580:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931586:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931592:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ee00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074757.931597:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931604:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931623:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074757.931626:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074757.931633:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931639:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52784 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931654:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931659:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931665:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931669:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0fc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074757.931672:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931677:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931684:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6510d4600) 00010000:00010000:1.0:1769074757.931686:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074757.931688:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931693:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd5278b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.931738:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5277d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb9 expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.931752:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb9 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.931759:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074757.931762:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074757.931765:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:0.0:1769074757.932268:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.932501:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.932533:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667776 00000100:00100000:3.0:1769074757.932541:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667776 00000100:00100000:3.0:1769074757.932596:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13655 00000100:00100000:3.0:1769074757.932605:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616667776:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.932618:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.932640:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd527a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afc7 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074757.932693:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057408:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074757.932738:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074757.932848:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.932863:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667648 00000100:00100000:3.0:1769074757.932867:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667648 00000100:00100000:3.0:1769074757.932898:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13656 00000100:00100000:3.0:1769074757.932905:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616667648:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.932917:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.932926:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afc0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.932940:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.932942:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.932954:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 13 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.932969:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 13 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.932981:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff2800, rc 0) 00000100:00100000:3.0:1769074757.933009:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616667648:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 105us (203us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.933016:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13656 00000100:00100000:3.0:1769074757.935886:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.935939:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616667904 00000100:00100000:3.0:1769074757.935948:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616667904 00000100:00100000:3.0:1769074757.936004:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13657 00000100:00100000:3.0:1769074757.936013:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616667904:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074757.936030:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074757.936046:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afce expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074757.936079:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.936085:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074757.936094:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 14 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.936119:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 14 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074757.936132:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3c00, rc 0) 00000100:00100000:3.0:1769074757.936176:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616667904:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 164us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.936183:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13657 00000100:00100000:3.0:1769074757.936315:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074757.936410:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616667008:12345-192.168.204.60@tcp:101:cat.0 Request processed in 26103us (26443us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074757.938359:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616668160 00000100:00100000:3.0:1769074757.938370:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13654 00000100:00100000:0.0:1769074757.938375:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616668160 00000100:00100000:0.0:1769074757.938449:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4606 00000100:00100000:0.0:1769074757.938460:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:62535:x1855007616668160:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074757.938774:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62535:x1855007616668160:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 316us (2459us total) trans 8589943822 rc 0/0 00000100:00100000:0.0:1769074757.938782:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4606 00000100:00100000:1.0:1769074757.940733:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62543:x1855007616667392:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 9596us (20111us total) trans 8589943821 rc 0/0 00000100:00100000:1.0:1769074757.940749:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13231 00002000:00010000:0.0:1769074757.940763:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074752 -> 1769074757 00002000:00010000:0.0:1769074757.940768:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074750 -> 1769074757 00002000:00010000:0.0:1769074757.940771:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074752 -> 1769074757 00000100:00100000:0.0:1769074757.940795:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057408:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074757.940859:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd527a0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc7 expref: 13 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074757.940887:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd527a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc7 expref: 13 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074757.940899:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb000, rc 0) 00000100:00100000:0.0:1769074757.940943:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616667776:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 8339us (8444us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074757.940953:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13655 00000100:00100000:3.0:1769074757.949499:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616668288 00000100:00100000:3.0:1769074757.949509:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616668288 00002000:00100000:3.0:1769074757.949566:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546c6b800 x1855007616668288/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.949577:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6b800 with x1855007616668288 ext(4194304->4808703) 00002000:00010000:3.0:1769074757.949582:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c6b800 00000100:00100000:3.0:1769074757.949611:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8273 00000100:00100000:3.0:1769074757.949621:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616668288:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074757.963856:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625057280:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074757.963891:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074757.963965:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616667520:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 39410us (40225us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074757.963972:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616668288:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14349us (14810us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074757.963981:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6751 00000100:00100000:3.0:1769074757.963986:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8273 00002000:00100000:3.0:1769074757.964013:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546c6b800 x1855007616668288/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.964027:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6b800 with x1855007616668288 ext(4194304->4808703) 00002000:00010000:3.0:1769074757.964035:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c6b800 00000100:00100000:3.0:1769074757.969113:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074757.969502:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616668544 00000100:00100000:1.0:1769074757.969515:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616668544 00000100:00100000:1.0:1769074757.969592:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13232 00000100:00100000:1.0:1769074757.969603:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62540:x1855007616668544:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074757.969624:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074757.969648:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd527b5 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afd5 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.971172:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971203:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971213:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971228:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971237:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971244:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971250:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074757.971254:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971262:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971274:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074757.971277:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074757.971281:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971285:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd527bc lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 13 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074757.971327:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd527b5 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 12 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afd5 expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.971338:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074757.971341:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074757.971345:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd527b5 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 12 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afd5 expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074757.971354:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e800, rc 0) 00000100:00100000:1.0:1769074757.971443:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62540:x1855007616668544:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1841us (2345us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074757.971453:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13232 00000100:00100000:3.0:1769074757.978042:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616668672 00000100:00100000:3.0:1769074757.978055:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616668672 00002000:00100000:3.0:1769074757.978110:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546c69880 x1855007616668672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/0 e 0 to 0 dl 1769074768 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074757.978123:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69880 with x1855007616668672 ext(1048576->2097151) 00002000:00010000:3.0:1769074757.978130:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c69880 00000100:00100000:3.0:1769074757.978233:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8274 00000100:00100000:3.0:1769074757.978244:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616668672:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074757.979021:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669056 00000100:00100000:1.0:1769074757.979029:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669056 00000100:00100000:1.0:1769074757.979069:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4607 00000100:00100000:1.0:1769074757.979077:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e9c0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62540:x1855007616669056:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074757.979258:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e9c0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:62540:x1855007616669056:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 181us (1171us total) trans 8589943823 rc 0/0 00000100:00100000:1.0:1769074757.979268:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4607 00000100:00100000:0.0:1769074757.979906:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616668800 00000100:00100000:0.0:1769074757.979923:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616668800 00000100:00100000:0.0:1769074757.979971:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13658 00000100:00100000:0.0:1769074757.979980:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616668800:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074757.979991:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074757.980270:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afdc expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074757.980296:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.980298:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074757.980309:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 20 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074757.980327:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 20 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074757.980340:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70000, rc 0) 00000100:00100000:0.0:1769074757.980380:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616668800:12345-192.168.204.60@tcp:101:cat.0 Request processed in 402us (2384us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074757.980387:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13658 00000100:00100000:3.0:1769074758.010620:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616668672:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32365us (32673us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074758.010636:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8274 00002000:00100000:3.0:1769074758.010652:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546c69880 x1855007616668672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:293/0 lens 488/440 e 0 to 0 dl 1769074768 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.010665:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69880 with x1855007616668672 ext(1048576->2097151) 00002000:00010000:3.0:1769074758.010671:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c69880 00000100:00100000:0.0:1769074758.011089:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.012377:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669312 00000100:00100000:3.0:1769074758.012393:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669312 00000100:00100000:3.0:1769074758.012470:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4608 00000100:00100000:3.0:1769074758.012479:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62529:x1855007616669312:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074758.012760:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:62529:x1855007616669312:12345-192.168.204.60@tcp:35:cat.0 Request processed in 282us (1671us total) trans 8589943824 rc 0/0 00000100:00100000:3.0:1769074758.012769:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4608 00000100:00100000:0.0:1769074758.028587:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.028838:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669440 00000100:00100000:3.0:1769074758.028852:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669440 00000100:00100000:3.0:1769074758.028932:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13659 00000100:00100000:3.0:1769074758.028942:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62539:x1855007616669440:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074758.028962:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.028984:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afe3 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.029454:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.029469:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074758.029471:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074758.029476:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.029514:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.029527:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 19 pid: 9048 timeout: 1845 lvb_type: 0 00000100:00100000:3.0:1769074758.029542:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057536:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.029743:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.040130:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057536:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.040192:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.040217:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b719800, rc 0) 00000100:00100000:3.0:1769074758.040317:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62539:x1855007616669440:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11376us (11772us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.040327:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13659 00000100:00100000:0.0:1769074758.043539:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669568 00000100:00100000:0.0:1769074758.043568:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669568 00002000:00100000:0.0:1769074758.043730:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1728511) req@ffff89b54d102300 x1855007616669568/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.043742:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d102300 with x1855007616669568 ext(1048576->1728511) 00010000:00010000:0.0:1769074758.043753:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 20 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074758.043771:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 20 pid: 9048 timeout: 1845 lvb_type: 0 00002000:00010000:0.0:1769074758.043780:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d102300 00000100:00100000:0.0:1769074758.043842:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 679 00000100:00100000:0.0:1769074758.043853:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616669568:12345-192.168.204.60@tcp:4:cat.0 00000100:00080000:1.0:1769074758.058854:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074758.059603:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616669568:12345-192.168.204.60@tcp:4:cat.0 Request processed in 15743us (21984us total) trans 21474841139 rc 0/0 00000100:00100000:3.0:1769074758.059616:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 679 00002000:00100000:3.0:1769074758.059631:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1728511) req@ffff89b54d102300 x1855007616669568/t21474841139(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:294/0 lens 488/448 e 0 to 0 dl 1769074769 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.059641:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d102300 with x1855007616669568 ext(1048576->1728511) 00010000:00010000:3.0:1769074758.059653:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:3.0:1769074758.059671:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00002000:00010000:3.0:1769074758.059678:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d102300 00000100:00100000:3.0:1769074758.066266:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.069084:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669696 00000100:00100000:0.0:1769074758.069100:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669696 00010000:00010000:0.0:1769074758.069159:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00000100:00100000:0.0:1769074758.069206:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6752 00000100:00100000:0.0:1769074758.069214:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616669696:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074758.069230:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074758.069251:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074758.069271:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074758.069294:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52745 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1728511) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af96 expref: 22 pid: 9048 timeout: 1845 lvb_type: 0 00010000:00010000:0.0:1769074758.069310:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.069313:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.069320:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.069331:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.069367:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074758.069381:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625057664:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074758.069463:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.078204:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625057664:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074758.078234:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.078299:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616669696:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9084us (12044us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.078313:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6752 00000100:00100000:1.0:1769074758.096712:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.096789:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669824 00000100:00100000:3.0:1769074758.096799:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669824 00002000:00100000:3.0:1769074758.096865:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b549208a80 x1855007616669824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.096877:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549208a80 with x1855007616669824 ext(1048576->2097151) 00002000:00010000:3.0:1769074758.096884:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549208a80 00000100:00100000:3.0:1769074758.096927:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8275 00000100:00100000:3.0:1769074758.096936:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616669824:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074758.142361:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616669824:12345-192.168.204.60@tcp:3:cat.0 Request processed in 45412us (45644us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074758.142387:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8275 00002000:00100000:3.0:1769074758.142401:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b549208a80 x1855007616669824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/440 e 0 to 0 dl 1769074769 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.142412:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549208a80 with x1855007616669824 ext(1048576->2097151) 00002000:00010000:3.0:1769074758.142418:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549208a80 00000100:00100000:1.0:1769074758.159323:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.159434:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616669952 00000100:00100000:3.0:1769074758.159445:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616669952 00000100:00100000:3.0:1769074758.159505:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13660 00000100:00100000:3.0:1769074758.159516:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616669952:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074758.159538:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.159566:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afea expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.159592:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.159594:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.159604:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 15 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.159624:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 15 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.159638:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718000, rc 0) 00000100:00100000:3.0:1769074758.159707:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616669952:12345-192.168.204.60@tcp:101:cat.0 Request processed in 193us (398us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.159714:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13660 00000100:00100000:1.0:1769074758.165856:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.166197:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616670080 00000100:00100000:3.0:1769074758.166213:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616670080 00000100:00100000:3.0:1769074758.166298:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13661 00000100:00100000:3.0:1769074758.166309:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616670080:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074758.166332:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.166356:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aff1 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.166401:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.166403:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.166415:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.166437:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.166451:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718600, rc 0) 00000100:00100000:3.0:1769074758.166513:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616670080:12345-192.168.204.60@tcp:101:cat.0 Request processed in 206us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.166521:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13661 00000100:00100000:3.0:1769074758.234959:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616670208 00000100:00100000:3.0:1769074758.234972:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616670208 00002000:00100000:3.0:1769074758.235030:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54759f800 x1855007616670208/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.235044:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f800 with x1855007616670208 ext(2097152->4194303) 00002000:00010000:3.0:1769074758.235052:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759f800 00000100:00100000:3.0:1769074758.235088:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8276 00000100:00100000:3.0:1769074758.235098:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616670208:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074758.242456:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616670336 00000100:00100000:0.0:1769074758.242466:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616670336 00002000:00100000:0.0:1769074758.242517:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b67b6d5c50 x1855007616670336/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.242529:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b6d5c50 with x1855007616670336 ext(2097152->4194303) 00002000:00010000:0.0:1769074758.242535:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b6d5c50 00000100:00100000:0.0:1769074758.242575:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8277 00000100:00100000:0.0:1769074758.242584:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b6d5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616670336:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074758.250263:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.250855:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616670592 00000100:00100000:3.0:1769074758.250871:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616670592 00000100:00100000:3.0:1769074758.251042:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13233 00000100:00100000:3.0:1769074758.251056:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:62548:x1855007616670592:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074758.251088:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.251120:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd527df lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879aff8 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.251203:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.251218:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.251232:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.251247:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.251254:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.251260:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.252567:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.252586:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.252596:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074758.252609:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074758.261952:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.261970:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.261976:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.261983:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.261988:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0fc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074758.261992:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262000:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262016:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074758.262019:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074758.262024:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262030:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd527e6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262044:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262050:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262060:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262065:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ee00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074758.262068:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262073:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262080:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543eab00) 00010000:00010000:3.0:1769074758.262082:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074758.262085:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262090:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd527ed lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.262149:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd527df lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 233 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.262163:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 232 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.262170:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074758.262172:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074758.262175:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ec00, rc 0) 00000100:00100000:3.0:1769074758.262263:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+232:62548:x1855007616670592:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11210us (12014us total) trans 8589943825 rc 0/0 00000100:00100000:3.0:1769074758.262274:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13233 00000100:00100000:1.0:1769074758.291464:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b6d5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616670336:12345-192.168.204.60@tcp:3:cat.0 Request processed in 48869us (49104us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074758.291477:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8277 00002000:00100000:1.0:1769074758.291492:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b67b6d5c50 x1855007616670336/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/440 e 0 to 0 dl 1769074769 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074758.291503:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b6d5c50 with x1855007616670336 ext(2097152->4194303) 00002000:00010000:1.0:1769074758.291510:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b6d5c50 00000100:00100000:1.0:1769074758.293613:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.294101:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616670720 00000100:00100000:3.0:1769074758.294111:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616670720 00000100:00100000:3.0:1769074758.294168:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13662 00000100:00100000:3.0:1769074758.294177:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62543:x1855007616670720:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074758.294193:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.294210:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879afff expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.294240:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.294249:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 15 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.294256:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 11 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.294263:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.294264:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.294268:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2195456->3088383] (req 2195456->3088383) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.294299:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 11 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.294310:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 11 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.294327:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057792:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.294424:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 15 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.294433:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 15 pid: 27844 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.294441:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057920:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.294472:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.294480:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 20 pid: 9048 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.294487:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625058048:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.294510:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.295850:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616670976 00000100:00100000:3.0:1769074758.295860:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616670976 00000100:00100000:3.0:1769074758.295907:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13234 00000100:00100000:3.0:1769074758.296057:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+232:62551:x1855007616670976:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074758.296078:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.296096:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52809 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b006 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.296203:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.296216:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074758.296226:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074758.296236:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074758.296329:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52809 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 235 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.296345:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 234 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.296353:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074758.296356:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1769074758.296359:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fc00, rc 0) 00000100:00100000:3.0:1769074758.296408:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62551:x1855007616670976:12345-192.168.204.60@tcp:101:cat.0 Request processed in 356us (1180us total) trans 8589943826 rc 0/0 00000100:00100000:3.0:1769074758.296415:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13234 00000100:00100000:1.0:1769074758.299787:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616670208:12345-192.168.204.60@tcp:3:cat.0 Request processed in 64686us (65203us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074758.299799:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8276 00002000:00100000:1.0:1769074758.299847:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54759f800 x1855007616670208/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/440 e 0 to 0 dl 1769074769 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074758.299858:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f800 with x1855007616670208 ext(2097152->4194303) 00002000:00010000:1.0:1769074758.299864:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759f800 00000100:00100000:3.0:1769074758.301235:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671104 00000100:00100000:3.0:1769074758.301247:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671104 00000100:00100000:3.0:1769074758.301305:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13663 00000100:00100000:3.0:1769074758.301314:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62548:x1855007616671104:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074758.301330:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.301348:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b00d expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.301377:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 15 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.301385:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.301393:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 11 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.301399:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.301401:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.301404:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1429504->1531903] (req 1429504->1531903) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.301434:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 11 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.301445:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 11 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.301457:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058176:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.301543:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.301563:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 22 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.301573:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058304:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.301605:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 15 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.301614:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 15 pid: 27843 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.301622:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058432:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.301642:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074758.309311:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057792:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074758.309330:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625057920:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.310337:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.310367:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.310388:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.310417:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671360 00000100:00100000:3.0:1769074758.310424:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671360 00000100:00100000:3.0:1769074758.310470:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13664 00000100:00100000:3.0:1769074758.310481:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616671360:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074758.310497:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.310515:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b014 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.310539:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.310541:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.310561:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.310583:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.310596:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:3.0:1769074758.310649:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616671360:12345-192.168.204.60@tcp:101:cat.0 Request processed in 170us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.310656:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13664 00000100:00100000:3.0:1769074758.310694:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671232 00000100:00100000:3.0:1769074758.310698:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671232 00010000:00010000:3.0:1769074758.310736:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.310762:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671488 00000100:00100000:3.0:1769074758.310765:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671488 00010000:00010000:3.0:1769074758.310778:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 16 pid: 27844 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.310797:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6753 00000100:00100000:3.0:1769074758.310804:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616671232:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.310833:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.310857:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.310869:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.310891:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd527ae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afce expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.310913:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.310934:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616671232:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 130us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.310942:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6753 00000100:00100000:3.0:1769074758.310991:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6754 00000100:00100000:3.0:1769074758.310998:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616671488:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.311006:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.311015:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 16 pid: 27844 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.311025:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 16 pid: 27844 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.311037:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd527d8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff1 expref: 16 pid: 27844 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.311046:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.311065:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616671488:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 67us (676us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.311070:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6754 00000100:00100000:0.0:1769074758.314139:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625058048:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.314192:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2195456->3088383] (req 2195456->3088383) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.314211:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f800, rc 0) 00000100:00100000:0.0:1769074758.314294:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62543:x1855007616670720:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20117us (20689us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.314305:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13662 00000100:00100000:3.0:1769074758.319246:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.319274:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.319294:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.319324:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671616 00000100:00100000:3.0:1769074758.319336:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671616 00010000:00010000:3.0:1769074758.319387:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 11 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.319420:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671744 00000100:00100000:3.0:1769074758.319424:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671744 00010000:00010000:3.0:1769074758.319437:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 24 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.319458:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616672000 00000100:00100000:3.0:1769074758.319461:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616672000 00010000:00010000:3.0:1769074758.319473:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 14 pid: 27843 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.319503:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6755 00000100:00100000:3.0:1769074758.319512:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616671616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.319530:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.319565:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 11 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.319578:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 11 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.319609:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.319674:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616671616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 162us (440us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.319684:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6755 00000100:00100000:3.0:1769074758.319701:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6756 00000100:00100000:3.0:1769074758.319708:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616671744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.319715:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.319723:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 24 pid: 9048 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.319732:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 24 pid: 9048 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.319745:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.319764:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616671744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 57us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.319770:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6756 00000100:00100000:3.0:1769074758.319781:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6757 00000100:00100000:3.0:1769074758.319787:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616672000:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.319795:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.319803:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 14 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.319853:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 14 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.319867:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.319869:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.319880:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1429504->1531903) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.319892:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1429504->1531903) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.319938:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1429504->1531903) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074758.319954:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625058560:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074758.320019:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00010000:00010000:3.0:1769074758.320075:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd527a7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afc0 expref: 10 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.320092:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058176:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.320106:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd527c3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afdc expref: 23 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.320116:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058304:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.320128:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd5275a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afa4 expref: 14 pid: 27843 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.320137:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058432:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.320172:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1531903) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.320188:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:3.0:1769074758.320219:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62548:x1855007616671104:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 18906us (21111us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.320227:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13663 00000100:00100000:3.0:1769074758.320317:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.320460:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616671872 00000100:00100000:3.0:1769074758.320466:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616671872 00010000:00010000:3.0:1769074758.320487:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 22 pid: 9048 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.320504:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6758 00000100:00100000:3.0:1769074758.320510:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616671872:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.320518:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.320534:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 22 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.320545:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 22 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.320571:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd52722 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af73 expref: 22 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.320581:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.320583:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.320589:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.320597:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.320619:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074758.320629:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625058688:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074758.320659:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604000 going to sleep for 11 seconds 00000100:00080000:0.0:1769074758.321155:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074758.328212:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.329789:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.331122:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616672128 00000100:00100000:0.0:1769074758.331131:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616672128 00000100:00100000:0.0:1769074758.331166:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13665 00000100:00100000:0.0:1769074758.331174:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62539:x1855007616672128:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074758.331186:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074758.331203:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b01b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.331231:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.331239:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074758.331241:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074758.331245:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.331277:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.331289:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074758.331302:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058816:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074758.331368:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984b40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074758.332511:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625058560:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074758.332538:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074758.332595:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616672000:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12807us (13299us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.332606:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6757 00000100:00100000:0.0:1769074758.332629:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625058688:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074758.332641:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074758.332673:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616671872:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12163us (12356us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.332680:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6758 00000100:00100000:0.0:1769074758.335711:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.335972:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616672384 00000100:00100000:3.0:1769074758.335984:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616672384 00002000:00100000:3.0:1769074758.336032:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2195456->2199551) req@ffff89b546f91f80 x1855007616672384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.336043:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91f80 with x1855007616672384 ext(2195456->2199551) 00010000:00010000:3.0:1769074758.336052:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.336068:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00002000:00010000:3.0:1769074758.336075:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f91f80 00000100:00100000:3.0:1769074758.336110:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 680 00000100:00100000:3.0:1769074758.336120:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616672384:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074758.336538:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616672384:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 417us (836us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074758.336546:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 680 00002000:00100000:3.0:1769074758.336577:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2195456->2199551) req@ffff89b546f91f80 x1855007616672384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/4536 e 0 to 0 dl 1769074769 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.336585:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91f80 with x1855007616672384 ext(2195456->2199551) 00010000:00010000:3.0:1769074758.336590:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.336601:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00002000:00010000:3.0:1769074758.336607:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f91f80 00000100:00100000:1.0:1769074758.338837:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.338891:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.343447:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625058816:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.343488:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.343509:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70c00, rc 0) 00000100:00100000:3.0:1769074758.348277:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.350124:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.350379:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616673024 00000100:00100000:3.0:1769074758.350388:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616673024 00000100:00100000:3.0:1769074758.350438:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4609 00000100:00100000:3.0:1769074758.350451:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62548:x1855007616673024:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074758.350667:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62548:x1855007616673024:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 220us (551us total) trans 8589943827 rc 0/0 00000100:00100000:3.0:1769074758.350676:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4609 00000100:00100000:1.0:1769074758.351732:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 681 00000100:00100000:0.0:1769074758.351789:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616672768:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074758.351992:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616672768:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 248us (3724us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074758.352000:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 681 00002000:00100000:0.0:1769074758.352011:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (3084288->3088383) req@ffff89b547599f80 x1855007616672768/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/4536 e 0 to 0 dl 1769074769 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.352026:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599f80 with x1855007616672768 ext(3084288->3088383) 00010000:00010000:0.0:1769074758.352035:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 14 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074758.352051:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 14 pid: 9048 timeout: 1846 lvb_type: 0 00002000:00010000:0.0:1769074758.352060:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547599f80 00000100:00100000:0.0:1769074758.369128:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.369266:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.369567:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616673536 00000100:00100000:3.0:1769074758.369579:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616673536 00000100:00100000:3.0:1769074758.370645:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4610 00000100:00100000:3.0:1769074758.370660:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62543:x1855007616673536:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074758.370906:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62543:x1855007616673536:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 252us (1640us total) trans 8589943828 rc 0/0 00000100:00100000:3.0:1769074758.370913:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4610 00000100:00100000:3.0:1769074758.389921:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.390300:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.394268:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616673152:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 24371us (25143us total) trans 21474841140 rc 0/0 00000100:00100000:1.0:1769074758.394279:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 682 00002000:00100000:1.0:1769074758.394290:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2195456->3088383) req@ffff89b546f92680 x1855007616673152/t21474841140(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/448 e 0 to 0 dl 1769074769 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074758.394297:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92680 with x1855007616673152 ext(2195456->3088383) 00010000:00010000:1.0:1769074758.394305:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:1.0:1769074758.394322:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00002000:00010000:1.0:1769074758.394330:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f92680 00000100:00100000:0.0:1769074758.404397:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.404537:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616673792 00000100:00100000:3.0:1769074758.404558:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616673792 00010000:00010000:3.0:1769074758.404598:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.404642:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6759 00000100:00100000:3.0:1769074758.404651:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616673792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.404666:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.404686:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.404699:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.404723:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52802 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2195456->3088383) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afff expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.404738:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.404740:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.404747:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.404756:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.404793:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074758.404851:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625058944:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074758.404929:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.416853:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625058944:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074758.416879:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.416936:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616673792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12285us (12551us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.416947:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6759 00000100:00100000:0.0:1769074758.418981:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.429610:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.437091:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.437135:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.437167:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.437214:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.438870:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616674176 00000100:00100000:3.0:1769074758.438883:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616674176 00000100:00100000:3.0:1769074758.438929:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616674304 00000100:00100000:3.0:1769074758.438933:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616674304 00000100:00100000:3.0:1769074758.438948:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616674432 00000100:00100000:3.0:1769074758.438951:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616674432 00000100:00100000:3.0:1769074758.438973:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13666 00000100:00100000:3.0:1769074758.438983:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616674176:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074758.439001:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.439019:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5284f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b029 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074758.439077:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625059072:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074758.439158:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.439177:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616674560 00000100:00100000:3.0:1769074758.439181:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616674560 00000100:00100000:3.0:1769074758.439217:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 977 00000100:00100000:3.0:1769074758.439225:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2406:x1855007616674560:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.439235:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.439257:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52848 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b022 expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.439274:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074758.439280:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.439307:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2406:x1855007616674560:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 83us (2093us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.439315:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 977 00000100:00100000:3.0:1769074758.439360:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13667 00000100:00100000:3.0:1769074758.439367:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616674304:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074758.439375:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.439383:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b030 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.439394:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.439397:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.439406:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 14 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.439422:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 14 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.439435:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e000, rc 0) 00000100:00100000:3.0:1769074758.439455:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616674304:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 89us (2320us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.439460:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13667 00000100:00100000:3.0:1769074758.439472:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13668 00000100:00100000:3.0:1769074758.439479:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616674432:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074758.439486:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.439494:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b037 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.439505:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.439507:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.439512:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 14 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.439522:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 14 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.439530:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f600, rc 0) 00000100:00100000:3.0:1769074758.439559:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2409:x1855007616674432:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 69us (2379us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.439570:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13668 00000100:00100000:0.0:1769074758.440059:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.443761:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616674688 00000100:00100000:3.0:1769074758.443777:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616674688 00002000:00100000:3.0:1769074758.443846:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b546f94000 x1855007616674688/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074758.443859:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94000 with x1855007616674688 ext(2097152->3145727) 00002000:00010000:3.0:1769074758.443865:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94000 00000100:00100000:3.0:1769074758.443901:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8284 00000100:00100000:3.0:1769074758.443910:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616674688:12345-192.168.204.60@tcp:3:cat.0 00002000:00010000:3.0:1769074758.452650:0:28954:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074757 -> 1769074758 00000100:00100000:3.0:1769074758.452685:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625059072:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074758.452730:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5284f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b029 expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.452755:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5284f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b029 expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074758.452768:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f800, rc 0) 00000100:00100000:3.0:1769074758.452875:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616674176:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 13892us (15793us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074758.452885:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13666 00000100:00100000:1.0:1769074758.462450:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.462716:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616674944 00000100:00100000:3.0:1769074758.462729:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616674944 00000100:00100000:3.0:1769074758.462789:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13236 00000100:00100000:3.0:1769074758.462798:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:62554:x1855007616674944:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074758.462854:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074758.462882:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52864 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b03e expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463011:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.463031:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afab expref: 17 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463039:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb9 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463047:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 234 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463054:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 6/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.463087:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 234 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463100:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 234 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.463114:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059200:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074758.463280:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616674688:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19369us (23218us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074758.463291:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8284 00002000:00100000:1.0:1769074758.463303:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b546f94000 x1855007616674688/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:294/0 lens 488/440 e 0 to 0 dl 1769074769 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074758.463313:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94000 with x1855007616674688 ext(2097152->3145727) 00002000:00010000:1.0:1769074758.463320:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94000 00010000:00010000:3.0:1769074758.463410:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb9 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463423:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52792 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afb9 expref: 21 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.463432:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059328:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.463465:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afab expref: 17 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.463473:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5276f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afab expref: 17 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.463481:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059456:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.463504:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074758.475713:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.475862:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616675072 00000100:00100000:1.0:1769074758.475874:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616675072 00010000:00010000:1.0:1769074758.475937:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 235 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:1.0:1769074758.475999:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6760 00000100:00100000:1.0:1769074758.476009:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:2409:x1855007616675072:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074758.476026:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074758.476032:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 235 pid: 6513 timeout: 1846 lvb_type: 0 00010000:00010000:1.0:1769074758.476045:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 235 pid: 6513 timeout: 1846 lvb_type: 0 00010000:00010000:1.0:1769074758.476068:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074758.476073:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074758.476086:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074758.476093:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074758.476151:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:2409:x1855007616675072:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 142us (447us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.476158:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6760 00010000:00010000:3.0:1769074758.476697:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd527f4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aff8 expref: 234 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.476718:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059200:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.476731:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059328:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074758.476740:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059456:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.476771:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074758.477581:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.477675:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.480218:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.480314:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616675456 00000100:00100000:0.0:1769074758.480331:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616675456 00000100:00100000:0.0:1769074758.480417:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13669 00000100:00100000:0.0:1769074758.480427:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62551:x1855007616675456:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074758.480451:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074758.480476:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b045 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.480509:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.480512:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.480522:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.480539:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.480565:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44000, rc 0) 00000100:00100000:0.0:1769074758.480644:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62551:x1855007616675456:12345-192.168.204.60@tcp:101:cat.0 Request processed in 218us (438us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.480652:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13669 00000100:00100000:0.0:1769074758.505303:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.515332:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.515379:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.515686:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616675712 00000100:00100000:1.0:1769074758.515699:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616675712 00000100:00100000:1.0:1769074758.515775:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13670 00000100:00100000:1.0:1769074758.515786:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616675712:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074758.515857:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074758.515879:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd5288e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b053 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074758.515942:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625059712:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074758.516039:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074758.525560:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.525684:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616675968 00000100:00100000:3.0:1769074758.525694:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616675968 00010000:00010000:3.0:1769074758.525740:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b03e expref: 19 pid: 6512 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.525796:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6763 00000100:00100000:3.0:1769074758.525835:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2407:x1855007616675968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.525858:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.525866:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b03e expref: 19 pid: 6512 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.525881:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b03e expref: 19 pid: 6512 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.525916:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074758.525923:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.525936:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.525944:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.525970:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.526030:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2407:x1855007616675968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 225us (490us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.526038:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6763 00010000:00010000:3.0:1769074758.526094:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5286b lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b03e expref: 18 pid: 6512 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.526112:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625059584:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074758.526136:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.526144:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.526151:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074758.526198:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:3.0:1769074758.526203:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074758.526222:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074758.526251:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52880 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.526266:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.526273:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074758.526276:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074758.526279:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97800, rc 0) 00000100:00100000:3.0:1769074758.526309:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62554:x1855007616675584:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 20808us (21018us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.526317:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13237 00000100:00100000:3.0:1769074758.555500:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.555560:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.555608:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.556710:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616676352 00000100:00100000:1.0:1769074758.556720:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616676352 00000100:00100000:1.0:1769074758.556765:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616676608 00000100:00100000:1.0:1769074758.556768:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616676608 00000100:00100000:1.0:1769074758.556830:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13238 00000100:00100000:1.0:1769074758.556840:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62557:x1855007616676352:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074758.556863:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074758.556890:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52895 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b05a expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074758.557011:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074758.557033:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 16 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074758.557042:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00000100:00100000:3.0:1769074758.557764:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625059840:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074758.557846:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074758.557900:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625059840 00000100:00100000:3.0:1769074758.557906:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625059840 00000100:00100000:3.0:1769074758.557937:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2641 00000100:00100000:3.0:1769074758.557942:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625059840:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074758.557964:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6389760 pending 0 free 1445400576 avail 1299161088 00000100:00100000:3.0:1769074758.557998:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625059840:12345-0@lo:41:osp-pre-0-1.0 Request processed in 56us (152us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.558004:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2641 00000100:00100000:3.0:1769074758.558043:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625059840:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074758.558118:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625059968:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074758.558144:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074758.558169:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625059968 00000100:00100000:3.0:1769074758.558173:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625059968 00000100:00100000:3.0:1769074758.558192:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2642 00000100:00100000:3.0:1769074758.558197:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625059968:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074758.558215:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6438912 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074758.558245:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625059968:12345-0@lo:41:osp-pre-1-0.0 Request processed in 47us (100us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.558250:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2642 00000100:00100000:3.0:1769074758.558275:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625059968:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1769074758.558333:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 16 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074758.558346:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 16 pid: 6512 timeout: 1846 lvb_type: 0 00000100:00100000:1.0:1769074758.558357:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625060096:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074758.558411:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074758.560265:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625060608:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.560372:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625060224:0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.567082:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074758.567161:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625061120:0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.567216:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074758.567296:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625061120 00000100:00100000:0.0:1769074758.567309:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625061120 00000100:00100000:0.0:1769074758.567404:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2281 00000100:00100000:0.0:1769074758.567418:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625061120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.567486:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625061120:12345-0@lo:400:kworker.0 Request processed in 71us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.567495:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2281 00000100:00080000:0.0:1769074758.567527:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074758.567586:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625061120:0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.567628:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625060608 00000100:00100000:1.0:1769074758.569311:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074758.569343:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625060736:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.569444:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074758.569473:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625061248:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.569649:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074758.569676:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1769074758.569682:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625060352:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.569708:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074758.569720:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625060864:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.569737:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074758.569747:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625061376:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.569765:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074758.569798:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625060224 00000100:00100000:1.0:1769074758.569802:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625060224 00000100:00100000:1.0:1769074758.569959:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1769074758.569965:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625060224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.570001:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625060224:12345-0@lo:400:kworker.0 Request processed in 37us (690us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.570007:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1769074758.570035:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625060992 00000100:00100000:1.0:1769074758.570039:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625060992 00000100:00100000:1.0:1769074758.570057:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625061504 00000100:00100000:1.0:1769074758.570059:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625061504 00000100:00100000:1.0:1769074758.570068:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625060736 00000100:00100000:1.0:1769074758.570070:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625060736 00000100:00100000:1.0:1769074758.570080:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625060352 00000100:00100000:1.0:1769074758.570082:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625060352 00000100:00100000:1.0:1769074758.570090:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625060864 00000100:00100000:1.0:1769074758.570092:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625060864 00000100:00100000:1.0:1769074758.570104:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2282 00000100:00100000:1.0:1769074758.570108:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625060992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.570137:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625060992:12345-0@lo:400:kworker.0 Request processed in 29us (1164us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.570142:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2282 00000100:00100000:1.0:1769074758.570154:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2283 00000100:00100000:1.0:1769074758.570160:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625061504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074758.570163:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2284 00000100:00100000:3.0:1769074758.570170:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625060736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.570185:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625061504:12345-0@lo:400:kworker.0 Request processed in 26us (1167us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.570190:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2283 00000100:00100000:3.0:1769074758.570196:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625060736:12345-0@lo:400:kworker.0 Request processed in 28us (752us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.570200:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2285 00000100:00100000:1.0:1769074758.570206:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625060352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074758.570206:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2284 00000100:00100000:3.0:1769074758.570217:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2286 00000100:00100000:3.0:1769074758.570222:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625060864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.570226:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625060352:12345-0@lo:400:kworker.0 Request processed in 22us (519us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.570231:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2285 00000100:00100000:3.0:1769074758.570243:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625060864:12345-0@lo:400:kworker.0 Request processed in 21us (505us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.570249:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625061248 00000100:00100000:3.0:1769074758.570249:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2286 00000100:00100000:1.0:1769074758.570252:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625061248 00000100:00100000:1.0:1769074758.570272:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625061376 00000100:00100000:1.0:1769074758.570275:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625061376 00000100:00100000:3.0:1769074758.570795:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1428 00000100:00100000:3.0:1769074758.570896:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+29:3651:x1855007625061248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074758.570940:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+29:3651:x1855007625061248:12345-0@lo:400:kworker.0 Request processed in 138us (1291us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.570947:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1428 00000100:00100000:3.0:1769074758.570963:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1429 00000100:00100000:3.0:1769074758.570969:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+32:3651:x1855007625061376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074758.570996:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+32:3651:x1855007625061376:12345-0@lo:400:kworker.0 Request processed in 28us (1230us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.571002:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1429 00000100:00100000:0.0:1769074758.571089:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625060608 00000100:00080000:0.0:1769074758.571105:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074758.571110:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074758.571112:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074758.571115:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074758.572048:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625060352:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.572659:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625060864:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.572674:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625061376:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.572685:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625060224:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.572697:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625060736:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.572709:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625061248:0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.572877:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1430 00000100:00100000:1.0:1769074758.572886:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625060608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074758.574170:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625060608:12345-0@lo:400:kworker.0 Request processed in 1283us (7100us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.574183:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1430 00000100:00080000:1.0:1769074758.574232:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074758.574281:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625060608:0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.574506:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616675840:12345-192.168.204.60@tcp:3:cat.0 Request processed in 58370us (59121us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074758.574515:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8285 00002000:00100000:0.0:1769074758.574526:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546f95f80 x1855007616675840/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:294/0 lens 488/440 e 0 to 0 dl 1769074769 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.574536:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95f80 with x1855007616675840 ext(2097152->4194303) 00002000:00010000:0.0:1769074758.574551:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f95f80 00000100:00100000:0.0:1769074758.577530:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625060480:0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.577562:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625060992:0@lo:400:kworker.0 00000100:00100000:0.0:1769074758.577574:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625061504:0@lo:400:kworker.0 00000100:00100000:3.0:1769074758.580245:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.580342:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.580436:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625060096:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074758.580470:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000100:00100000:0.0:1769074758.580562:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616676864 00000100:00100000:0.0:1769074758.580570:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616676864 00000100:00100000:0.0:1769074758.580623:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616676992 00000100:00100000:0.0:1769074758.580627:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616676992 00010000:00010000:0.0:1769074758.580719:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 17 pid: 6512 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074758.580756:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6764 00000100:00100000:0.0:1769074758.580765:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2408:x1855007616676992:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074758.580780:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074758.580786:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 17 pid: 6512 timeout: 1846 lvb_type: 0 00000100:00100000:1.0:1769074758.580791:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 978 00010000:00010000:0.0:1769074758.580799:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 17 pid: 6512 timeout: 1846 lvb_type: 0 00000100:00100000:1.0:1769074758.580799:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616676864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074758.580827:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074758.585108:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd528b8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b061 expref: 23 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074758.585265:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074758.585272:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074758.585286:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074758.585294:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074758.585457:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2408:x1855007616676864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4648us (5212us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.585468:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 978 00010000:00010000:0.0:1769074758.590186:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52887 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b04c expref: 17 pid: 6512 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074758.590211:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074758.590217:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074758.590228:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074758.590236:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074758.590269:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074758.590350:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074758.590358:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074758.590390:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074758.590513:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52895 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b05a expref: 237 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074758.590529:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b05a expref: 236 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074758.590536:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074758.590538:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074758.590540:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650800, rc 0) 00000100:00100000:0.0:1769074758.601412:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616676992:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 20632us (21054us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.601428:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6764 00000100:00100000:1.0:1769074758.602089:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:62557:x1855007616676352:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 45246us (46522us total) trans 8589943831 rc 0/0 00000100:00100000:1.0:1769074758.602099:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13238 00000100:00100000:0.0:1769074758.602228:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.602656:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616677120 00000100:00100000:1.0:1769074758.602668:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616677120 00000100:00100000:1.0:1769074758.602732:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13672 00000100:00100000:1.0:1769074758.602741:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616677120:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074758.602762:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074758.602784:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641650a00/0x7fe2323adfd528d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b06f expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074758.602871:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625061760:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074758.602949:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074758.618049:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.618904:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625061760:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074758.618946:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641650a00/0x7fe2323adfd528d4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b06f expref: 16 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074758.618973:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641650a00/0x7fe2323adfd528d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b06f expref: 16 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074758.618988:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650a00, rc 0) 00000100:00100000:1.0:1769074758.619040:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616677120:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16300us (16814us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074758.619049:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13672 00000100:00100000:0.0:1769074758.637657:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.648218:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.649683:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616677632 00000100:00100000:1.0:1769074758.649695:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616677632 00000100:00100000:1.0:1769074758.649765:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13673 00000100:00100000:1.0:1769074758.649774:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616677632:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074758.649792:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074758.649835:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd528f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b084 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074758.649902:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625062016:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074758.649974:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074758.650634:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.650664:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616677760 00000100:00100000:0.0:1769074758.650671:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616677760 00010000:00010000:0.0:1769074758.650698:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b05a expref: 236 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074758.650721:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6765 00000100:00100000:0.0:1769074758.650728:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:2407:x1855007616677760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074758.650740:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074758.650746:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b05a expref: 236 pid: 6513 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074758.650759:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd5289c lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b05a expref: 236 pid: 6513 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074758.650781:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074758.650786:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd528e9 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074758.650797:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd528e9 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 15 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074758.650824:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd528e9 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 15 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074758.650838:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074758.650867:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:2407:x1855007616677760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (235us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.650873:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6765 00000100:00100000:0.0:1769074758.671108:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.673108:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625062016:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074758.673150:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd528f0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b084 expref: 16 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074758.673171:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd528f0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b084 expref: 16 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074758.673188:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:1.0:1769074758.673249:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616677632:12345-192.168.204.60@tcp:101:cat.0 Request processed in 23476us (25040us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074758.673257:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13673 00000100:00100000:3.0:1769074758.679065:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616678016 00000100:00100000:3.0:1769074758.679079:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616678016 00000100:00100000:0.0:1769074758.681329:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.681508:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616678272 00000100:00100000:3.0:1769074758.681517:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616678272 00000100:00100000:3.0:1769074758.681579:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4611 00000100:00100000:3.0:1769074758.681589:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62539:x1855007616678272:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074758.682280:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4612 00000100:00100000:1.0:1769074758.682289:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62554:x1855007616678016:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074758.682401:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:62554:x1855007616678016:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 113us (11303us total) trans 8589943833 rc 0/0 00000100:00100000:1.0:1769074758.682407:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4612 00000100:00100000:0.0:1769074758.743762:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.748077:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616678400 00000100:00100000:1.0:1769074758.748102:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616678400 00000100:00100000:0.0:1769074758.752293:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13674 00000100:00100000:0.0:1769074758.752308:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62558:x1855007616678400:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074758.752337:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074758.752537:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b08b expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.752589:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 24 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.752598:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae8c expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.752606:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae69 expref: 16 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.752613:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.752615:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.752619:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1839104->2097151] (req 1839104->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.752652:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae69 expref: 16 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.752666:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae69 expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.752680:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062144:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.752823:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae8c expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.752834:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae8c expref: 13 pid: 27844 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.752843:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062272:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.752881:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 24 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.757667:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 24 pid: 28954 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.757688:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062400:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074758.757781:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984e40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074758.758905:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.758957:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:0.0:1769074758.760137:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd52554 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae69 expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.760153:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062144:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.760166:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd5258c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae8c expref: 13 pid: 27844 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.760180:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062272:192.168.204.60@tcp:104: 00000100:00080000:1.0:1769074758.760346:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074758.771535:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062400:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.771607:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1839104->2097151] (req 1839104->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 15 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.771632:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c600, rc 0) 00000100:00100000:0.0:1769074758.771715:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62558:x1855007616678400:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 19408us (27964us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.771725:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13674 00000100:00100000:0.0:1769074758.783447:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.783604:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616678784 00000100:00100000:3.0:1769074758.783616:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616678784 00010000:00010000:3.0:1769074758.783662:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 25 pid: 28954 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.783709:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6768 00000100:00100000:3.0:1769074758.783719:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616678784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.783736:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.783758:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 25 pid: 28954 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.783773:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 25 pid: 28954 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.783799:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd52515 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ae3f expref: 25 pid: 28954 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.783870:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.783873:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.783882:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 14 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.783902:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 14 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.783943:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 14 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074758.783965:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625062528:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074758.784046:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e7a6300 going to sleep for 11 seconds 00000100:00100000:1.0:1769074758.788451:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.789120:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625062528:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074758.789140:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.789174:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616678784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5456us (5748us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.789183:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6768 00000100:00100000:1.0:1769074758.800177:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.800283:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616679040 00000100:00100000:0.0:1769074758.800295:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616679040 00000100:00100000:0.0:1769074758.800353:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13675 00000100:00100000:0.0:1769074758.800364:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62558:x1855007616679040:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074758.800386:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074758.800408:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b092 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.800443:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd526f1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af57 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.800453:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.800467:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.800473:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.800476:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074758.800480:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2494463] (req 2097152->2494463) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.800515:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.800530:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 12 pid: 27843 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.800554:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062656:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.800653:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.800662:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.800670:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062784:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.800701:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd526f1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af57 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.800714:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd526f1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af57 expref: 23 pid: 8398 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074758.800722:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062912:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074758.800741:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984e40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.804175:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616679168 00000100:00100000:3.0:1769074758.804191:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616679168 00010000:00010000:3.0:1769074758.804255:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 13 pid: 27843 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.804322:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6769 00000100:00100000:3.0:1769074758.804333:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616679168:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.804353:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.804393:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 13 pid: 27843 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.804410:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 13 pid: 27843 timeout: 1846 lvb_type: 1 00010000:00010000:3.0:1769074758.804444:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.804506:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616679168:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 175us (544us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.804517:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6769 00010000:00010000:0.0:1769074758.805313:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45600/0x7fe2323adfd52737 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af88 expref: 12 pid: 27843 timeout: 1846 lvb_type: 1 00000100:00100000:0.0:1769074758.805334:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062656:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074758.805347:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062784:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074758.817857:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625062912:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074758.817912:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->2494463] (req 2097152->2494463) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074758.817933:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:0.0:1769074758.817998:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62558:x1855007616679040:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17635us (17830us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.818009:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13675 00000100:00100000:1.0:1769074758.825943:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616679296 00000100:00100000:1.0:1769074758.825957:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616679296 00010000:00010000:1.0:1769074758.825999:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:1.0:1769074758.826044:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6770 00000100:00100000:1.0:1769074758.826062:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616679296:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074758.826077:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074758.826101:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074758.826103:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074758.826118:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00010000:00010000:1.0:1769074758.826140:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd527d1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afea expref: 16 pid: 9048 timeout: 1846 lvb_type: 1 00010000:00010000:1.0:1769074758.826157:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074758.826214:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616679296:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 160us (1003us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074758.826222:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6770 00000100:00100000:0.0:1769074758.828535:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625063040:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074758.828584:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074758.828653:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616679424:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 2376us (2553us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074758.828669:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6771 00000100:00100000:3.0:1769074758.836964:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.837231:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616679552 00000100:00100000:0.0:1769074758.837248:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616679552 00002000:00100000:0.0:1769074758.837313:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2490368->2494463) req@ffff89b547598380 x1855007616679552/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.837325:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547598380 with x1855007616679552 ext(2490368->2494463) 00002000:00010000:0.0:1769074758.837331:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547598380 00000100:00100000:0.0:1769074758.837572:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8288 00000100:00100000:0.0:1769074758.837586:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616679552:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074758.837831:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616679552:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 248us (957us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074758.837839:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8288 00002000:00100000:0.0:1769074758.837846:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2490368->2494463) req@ffff89b547598380 x1855007616679552/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/4536 e 0 to 0 dl 1769074769 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.837854:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547598380 with x1855007616679552 ext(2490368->2494463) 00002000:00010000:0.0:1769074758.837857:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547598380 00000100:00100000:0.0:1769074758.851045:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.855044:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616679808 00000100:00100000:1.0:1769074758.855060:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616679808 00000100:00100000:1.0:1769074758.855149:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4613 00000100:00100000:1.0:1769074758.855161:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e9c1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62558:x1855007616679808:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074758.855796:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e9c1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62558:x1855007616679808:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 636us (4761us total) trans 8589943834 rc 0/0 00000100:00100000:1.0:1769074758.855828:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4613 00000100:00100000:0.0:1769074758.917265:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616679936 00000100:00100000:0.0:1769074758.917279:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616679936 00000100:00100000:0.0:1769074758.917383:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616680064 00000100:00100000:0.0:1769074758.917390:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616680064 00000100:00100000:0.0:1769074758.917443:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13676 00000100:00100000:0.0:1769074758.917455:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616679936:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074758.917478:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074758.917503:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd52905 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b099 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074758.917592:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625063168:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074758.917695:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074758.918222:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13677 00000100:00100000:1.0:1769074758.918234:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b35a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616680064:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074758.918244:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074758.918256:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd5290c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0a0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074758.918299:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625063296:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074758.918370:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00002000:00010000:0.0:1769074758.919675:0:27843:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074753 -> 1769074758 00002000:00010000:0.0:1769074758.919683:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074752 -> 1769074758 00002000:00010000:0.0:1769074758.919686:0:27843:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074753 -> 1769074758 00000100:00100000:0.0:1769074758.919708:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625063168:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074758.919745:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd52905 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b099 expref: 26 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.919767:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd52905 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b099 expref: 26 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074758.919780:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:0.0:1769074758.919894:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616679936:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2435us (2918us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074758.919904:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13676 00000100:00100000:0.0:1769074758.945411:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.955251:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074758.955415:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616680320 00000100:00100000:0.0:1769074758.955428:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616680320 00002000:00100000:0.0:1769074758.955493:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1839104->2097151) req@ffff89b54cf69500 x1855007616680320/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:294/0 lens 488/0 e 0 to 0 dl 1769074769 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074758.955505:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf69500 with x1855007616680320 ext(1839104->2097151) 00010000:00010000:0.0:1769074758.955516:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 15 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074758.955567:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 15 pid: 27843 timeout: 1846 lvb_type: 0 00002000:00010000:0.0:1769074758.955578:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cf69500 00000100:00100000:0.0:1769074758.955624:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 683 00000100:00100000:0.0:1769074758.955648:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616680320:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074758.980285:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074758.980336:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074758.980547:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616680704 00000100:00100000:3.0:1769074758.980560:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616680704 00010000:00010000:3.0:1769074758.980600:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 17 pid: 27843 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074758.980654:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6772 00000100:00100000:3.0:1769074758.980664:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616680704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074758.980682:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074758.980704:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 17 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.980718:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 17 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.980743:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd528f7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b08b expref: 17 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074758.980760:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.980762:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074758.980768:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52913 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0a7 expref: 23 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.980778:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52913 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0a7 expref: 23 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074758.980861:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52913 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0a7 expref: 23 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074758.980877:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625063552:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074758.980963:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e7a6300 going to sleep for 11 seconds 00000100:00100000:3.0:1769074758.985368:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625063552:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074758.985396:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074758.985451:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616680704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4788us (5113us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074758.985462:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6772 00000100:00100000:1.0:1769074759.006043:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.006139:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681088 00000100:00100000:3.0:1769074759.006151:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681088 00000100:00100000:3.0:1769074759.006226:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13679 00000100:00100000:3.0:1769074759.006236:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616681088:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074759.006256:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.006276:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5293d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0bc expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.006334:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625063680:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074759.006408:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00080000:1.0:1769074759.007042:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074759.013751:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.013800:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.013907:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681216 00000100:00100000:3.0:1769074759.013918:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681216 00000100:00100000:3.0:1769074759.013980:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616680832 00000100:00100000:3.0:1769074759.013984:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616680832 00000100:00100000:3.0:1769074759.014008:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13680 00000100:00100000:3.0:1769074759.014017:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616681216:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074759.014034:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.014053:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52944 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0c3 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.014075:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.014077:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.014087:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52944 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0c3 expref: 17 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.014105:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52944 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0c3 expref: 17 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.014119:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:3.0:1769074759.014175:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616681216:12345-192.168.204.60@tcp:101:cat.0 Request processed in 158us (434us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074759.014183:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13680 00000100:00100000:3.0:1769074759.014201:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13681 00000100:00100000:3.0:1769074759.014208:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62557:x1855007616680832:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074759.014215:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.014224:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0b5 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.014245:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.014254:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 11 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.014262:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 23 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.014268:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.014270:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.014273:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1048576->1978367] (req 1048576->1978367) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.014306:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 23 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.014318:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 23 pid: 27843 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074759.014331:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625063808:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074759.014387:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 11 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.014396:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 11 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:3.0:1769074759.014405:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625063936:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074759.014434:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.014449:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 16 pid: 9048 timeout: 1846 lvb_type: 0 00000100:00100000:3.0:1769074759.014458:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625064064:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074759.014479:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00080000:0.0:1769074759.014662:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074759.024316:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.024439:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681472 00000100:00100000:1.0:1769074759.024450:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681472 00000100:00100000:1.0:1769074759.024528:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13243 00000100:00100000:1.0:1769074759.024549:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+26:62570:x1855007616681472:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074759.024571:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.024592:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd52952 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0ca expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00000100:00080000:0.0:1769074759.024846:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1769074759.024964:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.024983:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 27 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.024994:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.025031:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 27 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.025044:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 27 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:1.0:1769074759.025059:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625064192:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074759.025140:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.025900:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:0.0:1769074759.028899:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:3.0:1769074759.030639:0:28954:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074757 -> 1769074758 00002000:00010000:3.0:1769074759.030649:0:28954:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074757 -> 1769074758 00000100:00100000:3.0:1769074759.030672:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625063680:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074759.030712:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5293d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0bc expref: 16 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.030736:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5293d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0bc expref: 16 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.030750:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e400, rc 0) 00000100:00100000:3.0:1769074759.030843:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616681088:12345-192.168.204.60@tcp:101:cat.0 Request processed in 24607us (24811us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074759.030852:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13679 00000100:00100000:1.0:1769074759.032802:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681728 00000100:00100000:1.0:1769074759.032846:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681728 00010000:00010000:1.0:1769074759.032891:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 24 pid: 27843 timeout: 1846 lvb_type: 1 00000100:00100000:1.0:1769074759.032919:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681600 00000100:00100000:1.0:1769074759.032923:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681600 00010000:00010000:1.0:1769074759.032935:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00000100:00100000:1.0:1769074759.032953:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6773 00000100:00100000:1.0:1769074759.032963:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616681728:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074759.032983:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074759.033003:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 24 pid: 27843 timeout: 1846 lvb_type: 1 00010000:00010000:1.0:1769074759.033017:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52825 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b014 expref: 24 pid: 27843 timeout: 1846 lvb_type: 1 00010000:00010000:1.0:1769074759.033046:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074759.033104:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616681728:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 141us (1947us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.033116:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6773 00000100:00100000:1.0:1769074759.033137:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6774 00000100:00100000:1.0:1769074759.033143:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616681600:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074759.033150:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074759.033159:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:1.0:1769074759.033170:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52856 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b030 expref: 12 pid: 10284 timeout: 1846 lvb_type: 1 00010000:00010000:1.0:1769074759.033183:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074759.033199:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616681600:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 56us (2017us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.033206:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6774 00000100:00100000:1.0:1769074759.043982:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.044097:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.044411:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681856 00000100:00100000:0.0:1769074759.044427:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681856 00010000:00010000:0.0:1769074759.044484:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 28 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074759.044523:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616681984 00000100:00100000:0.0:1769074759.044526:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616681984 00010000:00010000:0.0:1769074759.044552:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074759.044575:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6775 00000100:00100000:0.0:1769074759.044584:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2407:x1855007616681856:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.044601:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.044607:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 28 pid: 6513 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.044620:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 28 pid: 6513 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.044656:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074759.044662:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.044672:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.044679:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.044696:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.044761:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2407:x1855007616681856:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 177us (791us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.044768:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6775 00000100:00100000:0.0:1769074759.044787:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6776 00000100:00100000:0.0:1769074759.044793:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616681984:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.044803:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.044847:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.044858:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b719800/0x7fe2323adfd527ca lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879afe3 expref: 15 pid: 9048 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.044879:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.044883:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.044890:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.044900:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.044937:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.044953:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625064320:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074759.045007:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984240 going to sleep for 11 seconds 00010000:00010000:0.0:1769074759.045081:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5292f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ae expref: 27 pid: 6513 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074759.045098:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625064192:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074759.045124:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.045135:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.045141:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074759.045164:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074759.045497:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd52952 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 28 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.045513:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 27 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.045520:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074759.045523:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074759.045525:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650800, rc 0) 00000100:00100000:0.0:1769074759.045569:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+27:62570:x1855007616681472:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 21033us (21263us total) trans 8589943836 rc 0/0 00000100:00100000:0.0:1769074759.045577:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13243 00000100:00100000:0.0:1769074759.050740:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.056998:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616680960:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 28963us (31090us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074759.057016:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8289 00002000:00100000:3.0:1769074759.057033:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cf69180 x1855007616680960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.057049:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf69180 with x1855007616680960 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.057056:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf69180 00000100:00100000:1.0:1769074759.066651:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.067230:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616682240 00000100:00100000:3.0:1769074759.067240:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616682240 00002000:00100000:3.0:1769074759.067351:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1429504->2097151) req@ffff89b54cf69c00 x1855007616682240/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 504/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.067361:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf69c00 with x1855007616682240 ext(1429504->2097151) 00010000:00010000:3.0:1769074759.067368:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1531903) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 22 pid: 27844 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074759.067383:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5281e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1531903) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b00d expref: 22 pid: 27844 timeout: 1846 lvb_type: 0 00002000:00010000:3.0:1769074759.067390:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cf69c00 00000100:00100000:3.0:1769074759.067425:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 684 00000100:00100000:3.0:1769074759.067434:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616682240:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074759.069518:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625064704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074759.069558:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074759.069599:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625064576 00000100:00100000:1.0:1769074759.069607:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625064576 00000100:00100000:1.0:1769074759.069646:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625064704 00000100:00100000:1.0:1769074759.069649:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625064704 00000100:00100000:1.0:1769074759.069670:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1152 00000100:00100000:1.0:1769074759.069677:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+29:3649:x1855007625064576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074759.069701:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 976225600 pending 3829760 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074759.069754:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+29:3649:x1855007625064576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.069762:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1152 00000100:00100000:1.0:1769074759.069781:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1153 00000100:00100000:1.0:1769074759.069788:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+32:3651:x1855007625064704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074759.069797:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 421888 granted 964662592 pending 2605056 free 3900067840 avail 3668545536 00000100:00100000:1.0:1769074759.069848:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+32:3651:x1855007625064704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 60us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.069854:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1153 00000100:00100000:1.0:1769074759.069908:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625064704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074759.069938:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625064576:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1769074759.070005:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074759.070120:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074759.070129:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625064832:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074759.070158:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074759.070388:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625064832:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074759.081100:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.081492:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616682368 00000100:00100000:1.0:1769074759.081502:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616682368 00002000:00100000:1.0:1769074759.081559:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1974272->1978367) req@ffff89b547599180 x1855007616682368/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.081572:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599180 with x1855007616682368 ext(1974272->1978367) 00002000:00010000:1.0:1769074759.081578:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547599180 00000100:00100000:1.0:1769074759.081611:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8290 00000100:00100000:1.0:1769074759.081621:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616682368:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074759.081787:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616682368:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 167us (701us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074759.081796:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8290 00002000:00100000:1.0:1769074759.081803:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1974272->1978367) req@ffff89b547599180 x1855007616682368/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/4536 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.081846:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599180 with x1855007616682368 ext(1974272->1978367) 00002000:00010000:1.0:1769074759.081850:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547599180 00000100:00100000:0.0:1769074759.089644:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.104886:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.105287:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.105513:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625064960:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074759.105562:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 25 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.105587:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70400, rc 0) 00000100:00100000:0.0:1769074759.105648:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62565:x1855007616682496:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15772us (16014us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.105658:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13683 00000100:00100000:1.0:1769074759.109080:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616682880 00000100:00100000:1.0:1769074759.109096:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616682880 00000100:00100000:1.0:1769074759.109186:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4614 00000100:00100000:1.0:1769074759.109196:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62557:x1855007616682880:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074759.109452:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:62557:x1855007616682880:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 257us (4165us total) trans 8589943837 rc 0/0 00000100:00100000:1.0:1769074759.109460:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4614 00000100:00080000:1.0:1769074759.109483:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074759.116105:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.116394:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616683008 00000100:00100000:3.0:1769074759.116405:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616683008 00002000:00100000:3.0:1769074759.116459:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2494463) req@ffff89b54cf68380 x1855007616683008/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.116470:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68380 with x1855007616683008 ext(2097152->2494463) 00010000:00010000:3.0:1769074759.116480:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 17 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074759.116498:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 17 pid: 27843 timeout: 1846 lvb_type: 0 00002000:00010000:3.0:1769074759.116507:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cf68380 00000100:00100000:3.0:1769074759.116549:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 685 00000100:00100000:3.0:1769074759.116560:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616683008:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074759.130269:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.130445:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616683136 00000100:00100000:3.0:1769074759.130456:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616683136 00002000:00100000:3.0:1769074759.130506:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cf68e00 x1855007616683136/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.130519:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68e00 with x1855007616683136 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.130525:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf68e00 00000100:00100000:3.0:1769074759.130568:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8291 00000100:00100000:3.0:1769074759.130577:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616683136:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074759.153549:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616683136:12345-192.168.204.60@tcp:3:cat.0 Request processed in 22950us (23270us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074759.153566:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8291 00002000:00100000:3.0:1769074759.153582:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cf68e00 x1855007616683136/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.153592:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68e00 with x1855007616683136 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.153600:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf68e00 00000100:00100000:3.0:1769074759.154545:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616683008:12345-192.168.204.60@tcp:4:cat.0 Request processed in 37973us (38440us total) trans 17179874464 rc 0/0 00000100:00100000:3.0:1769074759.154557:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 685 00002000:00100000:3.0:1769074759.154566:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2494463) req@ffff89b54cf68380 x1855007616683008/t17179874464(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/448 e 0 to 0 dl 1769074770 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.154575:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68380 with x1855007616683008 ext(2097152->2494463) 00010000:00010000:3.0:1769074759.154585:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 19 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:3.0:1769074759.154601:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 19 pid: 27843 timeout: 1846 lvb_type: 0 00002000:00010000:3.0:1769074759.154608:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cf68380 00000100:00100000:0.0:1769074759.164036:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616683264 00000100:00100000:0.0:1769074759.164048:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616683264 00010000:00010000:0.0:1769074759.164093:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 19 pid: 27843 timeout: 1846 lvb_type: 0 00000100:00100000:0.0:1769074759.164143:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6778 00000100:00100000:0.0:1769074759.164152:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616683264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.164167:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.164188:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 19 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.164201:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 19 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.164228:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd528fe lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2494463) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b092 expref: 19 pid: 27843 timeout: 1846 lvb_type: 0 00010000:00010000:0.0:1769074759.164249:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.164252:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.164260:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 24 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.164271:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 24 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.164315:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 24 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.164330:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625065216:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074759.164442:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.172235:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.172793:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625065216:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074759.173058:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.173163:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616683264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9012us (9355us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.173173:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6778 00000100:00100000:3.0:1769074759.197656:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616683520 00000100:00100000:3.0:1769074759.197667:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616683520 00002000:00100000:3.0:1769074759.197719:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54759d880 x1855007616683520/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.197732:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759d880 with x1855007616683520 ext(2097152->4194303) 00002000:00010000:3.0:1769074759.197738:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759d880 00000100:00100000:3.0:1769074759.197774:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616683648 00000100:00100000:3.0:1769074759.197777:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616683648 00002000:00100000:3.0:1769074759.197794:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1978367) req@ffff89b54759dc00 x1855007616683648/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.197801:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759dc00 with x1855007616683648 ext(1048576->1978367) 00010000:00010000:3.0:1769074759.197828:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 23 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.197852:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 23 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.197853:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625065344:192.168.204.60@tcp:104: 00002000:00010000:3.0:1769074759.197860:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54759dc00 00010000:00010000:0.0:1769074759.197882:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd52975 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0df expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.197889:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 686 00000100:00100000:3.0:1769074759.197898:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616683648:12345-192.168.204.60@tcp:4:cat.0 00010000:00010000:0.0:1769074759.197899:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46800, rc 0) 00000100:00100000:0.0:1769074759.197948:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62568:x1855007616683392:12345-192.168.204.60@tcp:101:cat.0 Request processed in 25002us (25727us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.197956:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13684 00000100:00100000:3.0:1769074759.198049:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8292 00000100:00100000:3.0:1769074759.198056:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616683520:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074759.206487:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.206578:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616683776 00000100:00100000:0.0:1769074759.206589:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616683776 00000100:00100000:0.0:1769074759.206645:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13685 00000100:00100000:0.0:1769074759.206655:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62570:x1855007616683776:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074759.206673:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.206693:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0e6 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.206723:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52967 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d1 expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.206731:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.206733:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.206737:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1290240->1978367] (req 1290240->1978367) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.206772:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52967 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d1 expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.206785:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52967 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d1 expref: 17 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.206798:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625065472:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074759.212023:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.262412:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616683520:12345-192.168.204.60@tcp:3:cat.0 Request processed in 64344us (64829us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074759.262427:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8292 00002000:00100000:3.0:1769074759.262439:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54759d880 x1855007616683520/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.262450:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759d880 with x1855007616683520 ext(2097152->4194303) 00002000:00010000:3.0:1769074759.262456:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759d880 00000100:00100000:0.0:1769074759.263498:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616683648:12345-192.168.204.60@tcp:4:cat.0 Request processed in 65595us (65830us total) trans 21474841142 rc 0/0 00000100:00100000:0.0:1769074759.263508:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 686 00002000:00100000:0.0:1769074759.263517:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1978367) req@ffff89b54759dc00 x1855007616683648/t21474841142(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/448 e 0 to 0 dl 1769074770 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.263526:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759dc00 with x1855007616683648 ext(1048576->1978367) 00010000:00010000:0.0:1769074759.263563:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 25 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.263579:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd5294b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0b5 expref: 25 pid: 27844 timeout: 1847 lvb_type: 0 00002000:00010000:0.0:1769074759.263586:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54759dc00 00010000:00010000:1.0:1769074759.264562:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd52967 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d1 expref: 17 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.264586:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625065472:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074759.264618:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.264649:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45a00, rc 0) 00000100:00100000:1.0:1769074759.264703:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62570:x1855007616683776:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 58049us (58227us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.264714:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13685 00000100:00100000:3.0:1769074759.271251:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.274652:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.280653:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616684032 00000100:00100000:0.0:1769074759.280662:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616684032 00002000:00100000:0.0:1769074759.280709:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1290240->1294335) req@ffff89b547598000 x1855007616684032/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.280719:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547598000 with x1855007616684032 ext(1290240->1294335) 00002000:00010000:0.0:1769074759.280724:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547598000 00000100:00100000:1.0:1769074759.281121:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8293 00000100:00100000:1.0:1769074759.281133:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616684032:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074759.281291:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616684032:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 161us (10054us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074759.281298:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8293 00002000:00100000:1.0:1769074759.281314:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1290240->1294335) req@ffff89b547598000 x1855007616684032/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/4536 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.281324:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547598000 with x1855007616684032 ext(1290240->1294335) 00002000:00010000:1.0:1769074759.281328:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547598000 00000100:00100000:1.0:1769074759.284113:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.284158:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.286197:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13687 00000100:00100000:0.0:1769074759.286918:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616684416:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.286940:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.286959:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5298a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0f4 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.286982:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.286984:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.286993:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5298a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0f4 expref: 27 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.287009:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5298a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0f4 expref: 27 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.287022:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45800, rc 0) 00000100:00100000:0.0:1769074759.287084:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616684416:12345-192.168.204.60@tcp:101:cat.0 Request processed in 878us (2926us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.287091:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13687 00000100:00100000:3.0:1769074759.302546:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.303413:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616684544 00000100:00100000:1.0:1769074759.303425:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616684544 00002000:00100000:1.0:1769074759.303479:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54759a680 x1855007616684544/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.303491:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759a680 with x1855007616684544 ext(1048576->2097151) 00002000:00010000:1.0:1769074759.303497:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759a680 00000100:00100000:1.0:1769074759.303628:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8294 00000100:00100000:1.0:1769074759.303705:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616684544:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074759.309380:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616684672 00000100:00100000:3.0:1769074759.309390:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616684672 00002000:00100000:3.0:1769074759.309435:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54759d180 x1855007616684672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.309447:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759d180 with x1855007616684672 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.309453:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759d180 00000100:00100000:3.0:1769074759.309492:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8295 00000100:00100000:3.0:1769074759.309501:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616684672:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074759.324765:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616684672:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 15248us (15681us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074759.324782:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8295 00002000:00100000:3.0:1769074759.324799:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54759d180 x1855007616684672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.324840:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759d180 with x1855007616684672 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.324846:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759d180 00000100:00100000:0.0:1769074759.326491:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.326704:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616684544:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 23000us (24158us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074759.326717:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8294 00002000:00100000:1.0:1769074759.326728:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54759a680 x1855007616684544/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.326738:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759a680 with x1855007616684544 ext(1048576->2097151) 00002000:00010000:1.0:1769074759.326743:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759a680 00000100:00100000:1.0:1769074759.326767:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616684800 00000100:00100000:1.0:1769074759.326775:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616684800 00002000:00100000:1.0:1769074759.326821:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1974272->1978367) req@ffff89b5475d9180 x1855007616684800/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.326830:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d9180 with x1855007616684800 ext(1974272->1978367) 00002000:00010000:1.0:1769074759.326834:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475d9180 00000100:00100000:1.0:1769074759.326856:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8296 00000100:00100000:1.0:1769074759.326864:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616684800:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074759.326969:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616684800:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 106us (487us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074759.326975:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8296 00002000:00100000:1.0:1769074759.326980:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1974272->1978367) req@ffff89b5475d9180 x1855007616684800/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/4536 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.326988:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d9180 with x1855007616684800 ext(1974272->1978367) 00002000:00010000:1.0:1769074759.326991:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475d9180 00000100:00100000:0.0:1769074759.332078:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.332186:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616684928 00000100:00100000:3.0:1769074759.332199:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616684928 00002000:00100000:3.0:1769074759.332252:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5475db800 x1855007616684928/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.332264:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475db800 with x1855007616684928 ext(4194304->4808703) 00002000:00010000:3.0:1769074759.332270:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475db800 00000100:00100000:3.0:1769074759.332304:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8297 00000100:00100000:3.0:1769074759.332314:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616684928:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074759.341999:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616684928:12345-192.168.204.60@tcp:3:cat.0 Request processed in 9675us (9922us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074759.342014:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8297 00002000:00100000:3.0:1769074759.342028:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5475db800 x1855007616684928/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.342045:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475db800 with x1855007616684928 ext(4194304->4808703) 00002000:00010000:3.0:1769074759.342051:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475db800 00000100:00100000:0.0:1769074759.364449:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616685056 00000100:00100000:0.0:1769074759.364464:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616685056 00000100:00100000:0.0:1769074759.364574:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13688 00000100:00100000:0.0:1769074759.364586:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616685056:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.364611:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.364642:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52991 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b0fb expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.364727:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625065984:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074759.364861:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9849c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074759.378007:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616685184 00000100:00100000:0.0:1769074759.378016:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616685184 00000100:00100000:0.0:1769074759.378071:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13689 00000100:00100000:0.0:1769074759.378080:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62568:x1855007616685184:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.378102:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.378122:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52998 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b102 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.378146:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.378148:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.378157:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52998 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b102 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.378174:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52998 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b102 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.378188:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:0.0:1769074759.378252:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62568:x1855007616685184:12345-192.168.204.60@tcp:101:cat.0 Request processed in 172us (405us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.378259:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13689 00000100:00100000:1.0:1769074759.392200:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.392308:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616685312 00000100:00100000:0.0:1769074759.392319:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616685312 00000100:00100000:0.0:1769074759.392385:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13690 00000100:00100000:0.0:1769074759.392396:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616685312:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.392417:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.392439:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5299f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b109 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.392502:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625066112:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074759.392599:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.399159:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.399282:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616685568 00000100:00100000:0.0:1769074759.399294:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616685568 00000100:00100000:0.0:1769074759.399421:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13244 00000100:00100000:0.0:1769074759.399432:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:62575:x1855007616685568:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074759.399459:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.399478:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd529a6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b110 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.399600:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.399616:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074759.399630:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074759.399642:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074759.399760:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd529a6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 239 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.399790:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 238 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.399800:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074759.399804:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074759.399861:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d400, rc 0) 00000100:00100000:0.0:1769074759.399939:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+238:62575:x1855007616685568:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 508us (790us total) trans 8589943838 rc 0/0 00000100:00100000:0.0:1769074759.399947:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13244 00000100:00100000:3.0:1769074759.410963:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:0.0:1769074759.411103:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074758 -> 1769074759 00000100:00100000:0.0:1769074759.411130:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625066112:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074759.411169:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5299f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b109 expref: 26 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.411197:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5299f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b109 expref: 26 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.411210:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d200, rc 0) 00000100:00100000:0.0:1769074759.411289:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616685312:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18894us (19100us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074759.411299:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13690 00000100:00100000:1.0:1769074759.412088:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616685696 00000100:00100000:1.0:1769074759.412098:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616685696 00002000:00100000:1.0:1769074759.412150:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b547599c00 x1855007616685696/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.412167:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599c00 with x1855007616685696 ext(2097152->4194303) 00002000:00010000:1.0:1769074759.412173:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547599c00 00000100:00100000:1.0:1769074759.412210:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8298 00000100:00100000:1.0:1769074759.412219:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616685696:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074759.489960:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616685696:12345-192.168.204.60@tcp:3:cat.0 Request processed in 77654us (78922us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074759.489976:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8298 00002000:00100000:1.0:1769074759.489988:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b547599c00 x1855007616685696/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.490000:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599c00 with x1855007616685696 ext(2097152->4194303) 00002000:00010000:1.0:1769074759.490006:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547599c00 00000100:00100000:0.0:1769074759.510079:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.510587:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616685952 00000100:00100000:1.0:1769074759.510598:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616685952 00000100:00100000:1.0:1769074759.510664:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13245 00000100:00100000:1.0:1769074759.510675:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+27:62579:x1855007616685952:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074759.510698:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.510719:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd529bb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b117 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.510795:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.510839:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.510857:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.510873:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.510890:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.510896:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511021:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd529d0 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511033:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd529d0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511040:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd529d0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074759.511052:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074759.511144:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511151:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511157:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511163:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511168:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074759.511172:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511178:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511189:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074759.511192:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074759.511196:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511201:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd529c2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511214:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511219:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511224:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511230:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074759.511232:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511238:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511244:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294a00) 00010000:00010000:1.0:1769074759.511247:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074759.511249:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511258:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529c9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074759.511300:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd529bb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b117 expref: 30 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.511313:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd529d0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b117 expref: 29 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.511321:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074759.511323:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074759.511325:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9400, rc 0) 00000100:00100000:1.0:1769074759.511397:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62579:x1855007616685952:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 725us (1333us total) trans 8589943839 rc 0/0 00000100:00100000:1.0:1769074759.511405:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13245 00000100:00100000:3.0:1769074759.514553:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.514657:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616686080 00000100:00100000:1.0:1769074759.514666:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616686080 00002000:00100000:1.0:1769074759.514710:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f93b80 x1855007616686080/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.514721:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93b80 with x1855007616686080 ext(4194304->4808703) 00002000:00010000:1.0:1769074759.514726:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f93b80 00000100:00100000:1.0:1769074759.514759:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8299 00000100:00100000:1.0:1769074759.514768:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616686080:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074759.525391:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.526247:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616686336 00000100:00100000:1.0:1769074759.526262:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616686336 00000100:00100000:1.0:1769074759.526334:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4615 00000100:00100000:1.0:1769074759.526345:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62579:x1855007616686336:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074759.526637:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62579:x1855007616686336:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 292us (1254us total) trans 8589943840 rc 0/0 00000100:00100000:1.0:1769074759.526646:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4615 00000100:00100000:0.0:1769074759.534203:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616686464 00000100:00100000:0.0:1769074759.534221:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616686464 00000100:00100000:0.0:1769074759.534312:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 979 00000100:00100000:0.0:1769074759.534324:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2406:x1855007616686464:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.534341:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.534368:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd529d0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b117 expref: 30 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.534389:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074759.534393:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.534474:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2406:x1855007616686464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 152us (1412us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.534483:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 979 00000100:00100000:1.0:1769074759.537134:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616686080:12345-192.168.204.60@tcp:3:cat.0 Request processed in 22353us (22579us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074759.537149:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8299 00002000:00100000:1.0:1769074759.537162:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f93b80 x1855007616686080/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.537174:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93b80 with x1855007616686080 ext(4194304->4808703) 00002000:00010000:1.0:1769074759.537180:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f93b80 00000100:00100000:0.0:1769074759.560487:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616686720 00000100:00100000:0.0:1769074759.560498:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616686720 00000100:00100000:0.0:1769074759.560573:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4616 00000100:00100000:0.0:1769074759.560583:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62568:x1855007616686720:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:0.0:1769074759.560870:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62568:x1855007616686720:12345-192.168.204.60@tcp:35:cat.0 Request processed in 287us (494us total) trans 8589943841 rc 0/0 00000100:00100000:0.0:1769074759.561022:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4616 00000100:00100000:0.0:1769074759.563038:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616686848 00000100:00100000:0.0:1769074759.563049:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616686848 00000100:00100000:0.0:1769074759.563106:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13691 00000100:00100000:0.0:1769074759.563116:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62575:x1855007616686848:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074759.563135:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.563158:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b11e expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.563192:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.563204:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52998 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b102 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.563212:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.563214:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.563218:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2981888->3145727] (req 2981888->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.563253:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52998 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b102 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.563265:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52998 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b102 expref: 18 pid: 28954 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.563279:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625066240:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074759.563479:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.563489:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.563498:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625066368:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074759.563523:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.575536:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.575746:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625066240:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074759.575763:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625066368:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074759.575802:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2981888->3145727] (req 2981888->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.575864:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:0.0:1769074759.575939:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62575:x1855007616686848:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12825us (13053us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.575949:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13691 00000100:00100000:1.0:1769074759.583855:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625066496:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074759.583936:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074759.584163:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625066496 00000100:00100000:3.0:1769074759.584171:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625066496 00000100:00100000:3.0:1769074759.584231:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1155 00000100:00100000:3.0:1769074759.584241:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625066496:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074759.584266:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 973100352 pending 3026944 free 3900067840 avail 3668545536 00000100:00100000:3.0:1769074759.591539:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625066496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 7279us (7581us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074759.591556:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1155 00000100:00100000:3.0:1769074759.591661:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.594081:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687104 00000100:00100000:0.0:1769074759.594098:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687104 00010000:00010000:0.0:1769074759.594148:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.594190:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6782 00000100:00100000:0.0:1769074759.594199:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616687104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.594213:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.594232:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.594246:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.594269:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd5296e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0d8 expref: 26 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.594285:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.594288:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074759.594295:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.594305:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.594342:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.594369:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625066624:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074759.594449:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.596587:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625066496:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074759.599319:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625066624:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074759.599350:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.599483:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616687104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5283us (7822us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.599496:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6782 00000100:00100000:1.0:1769074759.602774:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687232 00000100:00100000:1.0:1769074759.602788:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687232 00000100:00100000:1.0:1769074759.602893:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13692 00000100:00100000:1.0:1769074759.602902:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62575:x1855007616687232:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074759.602919:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.602937:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b125 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.602966:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.602981:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 26 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.602989:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074759.602996:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.602998:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.603002:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->2703359] (req 2097152->2703359) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.603037:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074759.603050:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 10 pid: 10284 timeout: 1847 lvb_type: 1 00000100:00100000:1.0:1769074759.603066:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625066752:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074759.609176:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 26 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.609196:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 26 pid: 28954 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.609211:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625066880:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074759.616188:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.616209:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 15 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.616226:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625067008:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074759.616295:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.620702:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625066752:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074759.629609:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625066880:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074759.629736:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687488 00000100:00100000:3.0:1769074759.629745:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687488 00000100:00100000:3.0:1769074759.629840:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13693 00000100:00100000:3.0:1769074759.629850:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62579:x1855007616687488:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074759.629867:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.629894:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b12c expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.629927:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.629930:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.629936:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3899392->4194303] (req 3899392->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.629951:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3899392->4194303] (req 3899392->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.629966:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f800, rc 0) 00000100:00100000:3.0:1769074759.630024:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62579:x1855007616687488:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 175us (769us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074759.630031:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13693 00000100:00100000:0.0:1769074759.630261:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687360 00000100:00100000:0.0:1769074759.630268:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687360 00010000:00010000:0.0:1769074759.630297:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 11 pid: 10284 timeout: 1847 lvb_type: 1 00000100:00100000:0.0:1769074759.630333:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687616 00000100:00100000:0.0:1769074759.630336:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687616 00010000:00010000:0.0:1769074759.630349:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 27 pid: 28954 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.630372:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6783 00000100:00100000:0.0:1769074759.630380:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616687360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.630390:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.630412:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 11 pid: 10284 timeout: 1847 lvb_type: 1 00010000:00010000:0.0:1769074759.630426:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 11 pid: 10284 timeout: 1847 lvb_type: 1 00010000:00010000:0.0:1769074759.630451:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5285d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b037 expref: 11 pid: 10284 timeout: 1847 lvb_type: 1 00010000:00010000:0.0:1769074759.630466:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.630491:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616687360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 111us (1287us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.630497:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6783 00000100:00100000:0.0:1769074759.630511:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6784 00000100:00100000:0.0:1769074759.630516:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616687616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074759.630524:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074759.630542:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 27 pid: 28954 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.630552:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 27 pid: 28954 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.630564:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd52872 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b045 expref: 27 pid: 28954 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.630579:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.630598:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616687616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 82us (1268us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.630604:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6784 00000100:00100000:1.0:1769074759.631141:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.631406:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687744 00000100:00100000:3.0:1769074759.631413:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687744 00010000:00010000:3.0:1769074759.631438:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 17 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:3.0:1769074759.631467:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6785 00000100:00100000:3.0:1769074759.631475:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616687744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074759.631484:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074759.631496:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 17 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.631507:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 17 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.631535:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.631538:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.631546:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.631558:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.631566:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.631568:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.631571:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.631579:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.631617:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.631640:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625067136:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074759.631716:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.631726:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625067264:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074759.631747:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00010000:00010000:0.0:1769074759.632360:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd5282c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b01b expref: 17 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.632376:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625067008:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074759.632408:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.632420:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8200, rc 0) 00000100:00100000:0.0:1769074759.632467:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62575:x1855007616687232:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 29566us (29803us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074759.632476:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13692 00000100:00100000:3.0:1769074759.637444:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625067136:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074759.637461:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625067264:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074759.637480:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074759.637547:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616687744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6070us (6406us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074759.637559:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6785 00000100:00100000:1.0:1769074759.638519:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.638601:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616687872 00000100:00100000:3.0:1769074759.638610:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616687872 00002000:00100000:3.0:1769074759.638653:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3899392->3903487) req@ffff89b5444a4000 x1855007616687872/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.638664:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a4000 with x1855007616687872 ext(3899392->3903487) 00002000:00010000:3.0:1769074759.638670:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a4000 00000100:00100000:3.0:1769074759.638698:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8300 00000100:00100000:3.0:1769074759.638705:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616687872:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074759.638869:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616687872:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 163us (353us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074759.638876:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8300 00002000:00100000:3.0:1769074759.638883:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3899392->3903487) req@ffff89b5444a4000 x1855007616687872/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/4536 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.638892:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a4000 with x1855007616687872 ext(3899392->3903487) 00002000:00010000:3.0:1769074759.638895:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a4000 00000100:00100000:1.0:1769074759.650834:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688000 00000100:00100000:1.0:1769074759.650847:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688000 00002000:00100000:1.0:1769074759.650909:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2699264->2703359) req@ffff89b54759ea00 x1855007616688000/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.650922:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759ea00 with x1855007616688000 ext(2699264->2703359) 00002000:00010000:1.0:1769074759.650927:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759ea00 00000100:00100000:1.0:1769074759.650957:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8301 00000100:00100000:1.0:1769074759.650966:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616688000:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074759.651136:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616688000:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 170us (641us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074759.651143:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8301 00002000:00100000:1.0:1769074759.651149:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2699264->2703359) req@ffff89b54759ea00 x1855007616688000/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/4536 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.651158:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759ea00 with x1855007616688000 ext(2699264->2703359) 00002000:00010000:1.0:1769074759.651161:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759ea00 00000100:00100000:0.0:1769074759.660121:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.660339:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688128 00000100:00100000:1.0:1769074759.660349:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688128 00000100:00100000:1.0:1769074759.660404:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13694 00000100:00100000:1.0:1769074759.660412:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62579:x1855007616688128:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074759.660432:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.660449:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b133 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.660482:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 24 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.660491:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074759.660493:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074759.660497:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4509695] (req 4194304->4509695) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.660541:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 24 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.660555:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 24 pid: 27843 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.660569:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625067392:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074759.660666:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.666145:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625067392:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074759.666186:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4509695] (req 4194304->4509695) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.666205:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9000, rc 0) 00000100:00100000:1.0:1769074759.666268:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62579:x1855007616688128:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 5856us (6155us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.666277:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13694 00000100:00100000:1.0:1769074759.673244:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688384 00000100:00100000:1.0:1769074759.673254:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688384 00000100:00100000:1.0:1769074759.673304:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688512 00000100:00100000:1.0:1769074759.673307:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688512 00000100:00100000:1.0:1769074759.673319:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688640 00000100:00100000:1.0:1769074759.673320:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688640 00000100:00100000:1.0:1769074759.673338:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13695 00000100:00100000:1.0:1769074759.673344:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616688384:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074759.673358:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.673373:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd529fa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b13a expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074759.673417:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625067520:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074759.673486:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.673504:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13696 00000100:00100000:1.0:1769074759.673510:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616688512:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074759.673517:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.673542:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8400/0x7fe2323adfd52a01 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b141 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074759.673575:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625067648:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074759.673601:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074759.673681:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13697 00000100:00100000:0.0:1769074759.673691:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616688640:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.673696:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.673702:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52a08 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b148 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.673721:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625067776:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074759.673740:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.674307:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688256 00000100:00100000:3.0:1769074759.674312:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688256 00002000:00100000:3.0:1769074759.674336:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2981888->3145727) req@ffff89b54759f800 x1855007616688256/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.674343:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f800 with x1855007616688256 ext(2981888->3145727) 00010000:00010000:3.0:1769074759.674348:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.674362:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00002000:00010000:3.0:1769074759.674368:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54759f800 00000100:00100000:3.0:1769074759.674388:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 687 00000100:00100000:3.0:1769074759.674392:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616688256:12345-192.168.204.60@tcp:4:multiop.0 00002000:00010000:3.0:1769074759.681395:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074757 -> 1769074759 00002000:00010000:3.0:1769074759.681404:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074756 -> 1769074759 00002000:00010000:3.0:1769074759.681406:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074757 -> 1769074759 00000100:00100000:3.0:1769074759.681429:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625067776:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074759.681472:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52a08 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b148 expref: 28 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.681494:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52a08 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b148 expref: 28 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.681509:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:3.0:1769074759.681581:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616688640:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7891us (8415us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074759.681590:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13697 00002000:00010000:1.0:1769074759.681685:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074758 -> 1769074759 00002000:00010000:1.0:1769074759.681689:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074758 -> 1769074759 00000100:00100000:1.0:1769074759.681699:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625067520:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074759.681716:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd529fa lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b13a expref: 26 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074759.681729:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd529fa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b13a expref: 26 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074759.681735:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:1.0:1769074759.681752:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616688384:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8409us (8647us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074759.681757:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13695 00002000:00010000:0.0:1769074759.682469:0:27843:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074758 -> 1769074759 00002000:00010000:0.0:1769074759.682475:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074758 -> 1769074759 00002000:00010000:0.0:1769074759.682477:0:27843:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074758 -> 1769074759 00000100:00100000:0.0:1769074759.682488:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625067648:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074759.682509:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8400/0x7fe2323adfd52a01 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b141 expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.682536:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8400/0x7fe2323adfd52a01 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b141 expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.682546:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8400, rc 0) 00000100:00100000:0.0:1769074759.682603:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616688512:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9092us (9466us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074759.682611:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13696 00000100:00100000:3.0:1769074759.690222:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616688256:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 15817us (17155us total) trans 17179874465 rc 0/0 00000100:00100000:3.0:1769074759.690236:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 687 00002000:00100000:3.0:1769074759.690248:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2981888->3145727) req@ffff89b54759f800 x1855007616688256/t17179874465(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/448 e 0 to 0 dl 1769074770 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.690259:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f800 with x1855007616688256 ext(2981888->3145727) 00010000:00010000:3.0:1769074759.690269:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.690288:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00002000:00010000:3.0:1769074759.690296:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54759f800 00000100:00100000:3.0:1769074759.692355:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.692545:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688768 00000100:00100000:0.0:1769074759.692558:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688768 00000100:00100000:0.0:1769074759.692632:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13698 00000100:00100000:0.0:1769074759.692645:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62583:x1855007616688768:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.692668:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.692694:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b14f expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.692734:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.692744:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074759.692746:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074759.692752:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.692792:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.692860:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 15 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.692884:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625067904:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074759.692970:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9849c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.701197:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.701240:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.704643:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616688896 00000100:00100000:1.0:1769074759.704665:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616688896 00010000:00010000:1.0:1769074759.704721:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.704780:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6786 00000100:00100000:1.0:1769074759.704789:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616688896:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074759.704837:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074759.704860:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.704877:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.704903:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd529de lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2981888->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b11e expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.704919:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.704921:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.704929:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.704940:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.704981:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074759.704996:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625068032:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074759.705079:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:0.0:1769074759.710828:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625067904:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074759.710864:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.710883:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:1.0:1769074759.711012:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62583:x1855007616688768:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18368us (18667us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.711022:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13698 00000100:00100000:1.0:1769074759.713041:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625068032:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074759.713066:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074759.713119:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616688896:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8330us (11933us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.713130:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6786 00000100:00100000:0.0:1769074759.718113:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.718882:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616689152 00000100:00100000:1.0:1769074759.718891:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616689152 00002000:00100000:1.0:1769074759.718924:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4505600->4509695) req@ffff89b5475d8a80 x1855007616689152/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.718932:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8a80 with x1855007616689152 ext(4505600->4509695) 00002000:00010000:1.0:1769074759.718937:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475d8a80 00000100:00100000:1.0:1769074759.718957:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8302 00000100:00100000:1.0:1769074759.718962:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616689152:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074759.719182:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616689152:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 217us (1081us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074759.719190:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8302 00002000:00100000:1.0:1769074759.719196:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4505600->4509695) req@ffff89b5475d8a80 x1855007616689152/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/4536 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074759.719205:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8a80 with x1855007616689152 ext(4505600->4509695) 00002000:00010000:1.0:1769074759.719208:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475d8a80 00000100:00100000:3.0:1769074759.722094:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616689024:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 16763us (20842us total) trans 21474841143 rc 0/0 00000100:00100000:3.0:1769074759.722109:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 688 00002000:00100000:3.0:1769074759.722121:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1290240->1978367) req@ffff89b546f90a80 x1855007616689024/t21474841143(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/448 e 0 to 0 dl 1769074770 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.722131:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90a80 with x1855007616689024 ext(1290240->1978367) 00010000:00010000:3.0:1769074759.722142:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 18 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.722161:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 18 pid: 27844 timeout: 1847 lvb_type: 0 00002000:00010000:3.0:1769074759.722170:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f90a80 00000100:00100000:3.0:1769074759.724998:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.725595:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616689280 00000100:00100000:1.0:1769074759.725606:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616689280 00010000:00010000:1.0:1769074759.725668:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 18 pid: 27844 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.725719:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6787 00000100:00100000:1.0:1769074759.725730:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616689280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074759.725749:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074759.725774:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 18 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.725790:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 18 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.726174:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5297c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1290240->1978367) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0e6 expref: 18 pid: 27844 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.726193:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.726195:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.726203:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.726215:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.726256:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074759.726270:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625068160:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074759.726356:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.738390:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625068160:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074759.738431:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074759.738501:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616689280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12772us (13514us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.738511:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6787 00000100:00100000:3.0:1769074759.743960:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616689408 00000100:00100000:3.0:1769074759.743972:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616689408 00002000:00100000:3.0:1769074759.744024:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cf6b480 x1855007616689408/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.744036:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf6b480 with x1855007616689408 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.744042:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf6b480 00000100:00100000:3.0:1769074759.744068:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616689536 00000100:00100000:3.0:1769074759.744072:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616689536 00002000:00100000:3.0:1769074759.744086:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54cf69f80 x1855007616689536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.744094:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf69f80 with x1855007616689536 ext(1048576->2097151) 00002000:00010000:3.0:1769074759.744097:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf69f80 00000100:00100000:3.0:1769074759.744121:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8303 00000100:00100000:3.0:1769074759.744130:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616689408:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074759.744842:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8304 00000100:00100000:0.0:1769074759.744853:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616689536:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074759.816452:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.818356:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.818402:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.818894:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13700 00000100:00100000:0.0:1769074759.818904:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616690048:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074759.818913:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.818923:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd52a40 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b164 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074759.818953:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625068416:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074759.818987:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9849c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074759.821792:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.821893:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.822032:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616690176 00000100:00100000:3.0:1769074759.822042:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616690176 00000100:00100000:3.0:1769074759.822089:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616690304 00000100:00100000:3.0:1769074759.822092:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616690304 00000100:00100000:3.0:1769074759.822110:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13701 00000100:00100000:3.0:1769074759.822119:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616690176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074759.822135:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.822152:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b16b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.822176:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.822178:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.822187:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.822203:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.822217:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f200, rc 0) 00000100:00100000:3.0:1769074759.822265:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616690176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 147us (483us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074759.822272:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13701 00000100:00100000:3.0:1769074759.822288:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13702 00000100:00100000:3.0:1769074759.822295:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616690304:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074759.822302:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.822309:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a4e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b172 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.822364:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625068544:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074759.822414:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.825648:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.825855:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616690432 00000100:00100000:1.0:1769074759.825866:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616690432 00000100:00100000:1.0:1769074759.825929:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 980 00000100:00100000:1.0:1769074759.825938:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2409:x1855007616690432:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074759.825953:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074759.826339:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd52a2b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b156 expref: 32 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.826351:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074759.826356:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074759.827241:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625068416:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074759.827271:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd52a40 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b164 expref: 27 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.827285:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd52a40 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b164 expref: 27 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074759.827296:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877400, rc 0) 00000100:00100000:3.0:1769074759.828173:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625068544:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074759.828196:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a4e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b172 expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.828216:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a4e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b172 expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074759.828226:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:3.0:1769074759.828258:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616690304:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5963us (6366us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074759.828268:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13702 00000100:00100000:1.0:1769074759.830102:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2409:x1855007616690432:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4163us (4461us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.830113:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 980 00000100:00100000:0.0:1769074759.831880:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616690048:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12965us (13464us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074759.831894:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13700 00000100:00100000:3.0:1769074759.835064:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074759.835306:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616690560 00000100:00100000:0.0:1769074759.835317:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616690560 00000100:00100000:0.0:1769074759.835380:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13703 00000100:00100000:0.0:1769074759.835390:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62589:x1855007616690560:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074759.835415:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074759.835446:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b179 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.835481:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.835489:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074759.835492:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074759.835496:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3072000->3145727] (req 3072000->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.835539:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074759.835552:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 18 pid: 27843 timeout: 1847 lvb_type: 0 00000100:00100000:0.0:1769074759.835567:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625068672:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074759.835662:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9849c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074759.851500:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625068672:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074759.851551:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3072000->3145727] (req 3072000->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.851578:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:3.0:1769074759.851646:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62589:x1855007616690560:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 16257us (16592us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074759.851655:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13703 00000100:00100000:0.0:1769074759.851671:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8305 00000100:00100000:3.0:1769074759.851682:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616690688 00000100:00100000:0.0:1769074759.851683:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616690816:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074759.851687:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616690688 00000100:00100000:1.0:1769074759.852355:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13704 00000100:00100000:1.0:1769074759.852375:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62583:x1855007616690688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074759.852387:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.852405:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b180 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.852429:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074759.852435:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3072000->3145727] (req 3072000->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 28 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.852490:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:1.0:1769074759.852496:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.852555:0:27843:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3072000->3145727] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 28 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.852570:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.852581:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:1.0:1769074759.852628:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62583:x1855007616690688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 264us (1582us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.852635:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13704 00000100:00100000:0.0:1769074759.862515:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616690944:12345-192.168.204.60@tcp:4:cat.0 Request processed in 10936us (11240us total) trans 17179874466 rc 0/0 00000100:00100000:0.0:1769074759.862541:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 689 00002000:00100000:0.0:1769074759.862558:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4509695) req@ffff89b6491ee680 x1855007616690944/t17179874466(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/448 e 0 to 0 dl 1769074770 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.862569:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ee680 with x1855007616690944 ext(4194304->4509695) 00010000:00010000:0.0:1769074759.862581:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 21 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.862602:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 21 pid: 27843 timeout: 1847 lvb_type: 0 00002000:00010000:0.0:1769074759.862616:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b6491ee680 00000100:00100000:1.0:1769074759.871113:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691072 00000100:00100000:1.0:1769074759.871127:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691072 00010000:00010000:1.0:1769074759.871174:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 21 pid: 27843 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.871223:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6788 00000100:00100000:1.0:1769074759.871232:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616691072:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074759.871250:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074759.871272:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 21 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.871286:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 21 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.871311:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd529f3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4194304->4509695) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b133 expref: 21 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074759.871330:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.871332:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074759.871340:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.871351:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.871398:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.871411:0:33102:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 27 pid: 28955 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074759.871425:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625068928:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074759.871725:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:0.0:1769074759.871970:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616690816:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20288us (20772us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074759.871980:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8305 00002000:00100000:0.0:1769074759.871991:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b6491edc00 x1855007616690816/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.872001:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491edc00 with x1855007616690816 ext(1048576->2097151) 00002000:00010000:0.0:1769074759.872006:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491edc00 00000100:00100000:1.0:1769074759.881714:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625068928:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074759.881751:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074759.881852:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616691072:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10619us (10862us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.881863:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6788 00000100:00100000:0.0:1769074759.892070:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691200 00000100:00100000:0.0:1769074759.892081:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691200 00002000:00100000:0.0:1769074759.892137:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3072000->3145727) req@ffff89b6491eed80 x1855007616691200/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.892148:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491eed80 with x1855007616691200 ext(3072000->3145727) 00010000:00010000:0.0:1769074759.892158:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 28 pid: 28955 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.892174:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 28 pid: 28955 timeout: 1847 lvb_type: 0 00002000:00010000:0.0:1769074759.892183:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b6491eed80 00000100:00100000:0.0:1769074759.892221:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 690 00000100:00100000:0.0:1769074759.892230:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616691200:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074759.907131:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074759.907915:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691328 00000100:00100000:1.0:1769074759.907927:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691328 00000100:00100000:1.0:1769074759.907986:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13705 00000100:00100000:1.0:1769074759.907994:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62590:x1855007616691328:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074759.908011:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074759.908028:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b187 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.908055:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074759.908062:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.908074:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074759.908086:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9000, rc 0) 00000100:00100000:1.0:1769074759.908148:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62590:x1855007616691328:12345-192.168.204.60@tcp:101:cat.0 Request processed in 155us (1027us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.908155:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13705 00000100:00100000:0.0:1769074759.909688:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616691200:12345-192.168.204.60@tcp:4:cat.0 Request processed in 17446us (17764us total) trans 17179874467 rc 0/0 00000100:00100000:0.0:1769074759.909705:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 690 00002000:00100000:0.0:1769074759.909718:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3072000->3145727) req@ffff89b6491eed80 x1855007616691200/t17179874467(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/448 e 0 to 0 dl 1769074770 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.909730:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491eed80 with x1855007616691200 ext(3072000->3145727) 00010000:00010000:0.0:1769074759.909741:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 30 pid: 28955 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074759.909759:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 30 pid: 28955 timeout: 1847 lvb_type: 0 00002000:00010000:0.0:1769074759.909767:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b6491eed80 00000100:00100000:1.0:1769074759.915886:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074759.916054:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691456 00000100:00100000:3.0:1769074759.916067:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691456 00010000:00010000:3.0:1769074759.916116:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 30 pid: 28955 timeout: 1847 lvb_type: 0 00000100:00100000:3.0:1769074759.916161:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6789 00000100:00100000:3.0:1769074759.916172:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616691456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074759.916188:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074759.916209:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 30 pid: 28955 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.916223:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 30 pid: 28955 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.916248:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876a00/0x7fe2323adfd52a55 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3072000->3145727) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b179 expref: 30 pid: 28955 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.916264:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.916266:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.916273:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.916282:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.916291:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.916292:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074759.916295:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.916302:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.916343:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.916359:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625069056:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074759.916456:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074759.916466:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625069184:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074759.916491:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074759.918557:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625069056:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074759.918573:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625069184:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074759.918595:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074759.918652:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616691456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 2480us (2787us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074759.918663:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6789 00000100:00100000:0.0:1769074759.925932:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691584 00000100:00100000:0.0:1769074759.925942:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691584 00002000:00100000:0.0:1769074759.925991:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b6491ef100 x1855007616691584/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.926001:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ef100 with x1855007616691584 ext(2097152->3145727) 00002000:00010000:0.0:1769074759.926006:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ef100 00000100:00100000:0.0:1769074759.926038:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8306 00000100:00100000:0.0:1769074759.926047:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616691584:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074759.944403:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691712 00000100:00100000:0.0:1769074759.944413:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691712 00002000:00100000:0.0:1769074759.944461:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b6491ee300 x1855007616691712/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.944473:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ee300 with x1855007616691712 ext(2097152->4194303) 00002000:00010000:0.0:1769074759.944479:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ee300 00000100:00100000:0.0:1769074759.944505:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691840 00000100:00100000:0.0:1769074759.944509:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691840 00002000:00100000:0.0:1769074759.944535:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2703360->3899391) req@ffff89b6491ec000 x1855007616691840/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/0 e 0 to 0 dl 1769074770 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.944544:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ec000 with x1855007616691840 ext(2703360->3899391) 00002000:00010000:0.0:1769074759.944548:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491ec000 00000100:00100000:0.0:1769074759.944565:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8307 00000100:00100000:0.0:1769074759.944575:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616691712:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074759.947163:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8308 00000100:00100000:3.0:1769074759.947181:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616691840:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074759.967057:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616691584:12345-192.168.204.60@tcp:3:cat.0 Request processed in 40999us (41249us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074759.967074:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8306 00002000:00100000:0.0:1769074759.967089:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b6491ef100 x1855007616691584/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.967100:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ef100 with x1855007616691584 ext(2097152->3145727) 00002000:00010000:0.0:1769074759.967106:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ef100 00000100:00100000:3.0:1769074759.990005:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616691968 00000100:00100000:3.0:1769074759.990015:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616691968 00000100:00100000:3.0:1769074759.990069:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13706 00000100:00100000:3.0:1769074759.990079:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62590:x1855007616691968:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074759.990098:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074759.990115:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b18e expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.990144:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.990153:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074759.990155:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074759.990159:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.990193:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074759.990206:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 27 pid: 27843 timeout: 1847 lvb_type: 0 00000100:00100000:3.0:1769074759.990219:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625069312:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074759.990296:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074759.994070:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616691712:12345-192.168.204.60@tcp:3:cat.0 Request processed in 49486us (49915us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074759.994087:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8307 00002000:00100000:0.0:1769074759.994103:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b6491ee300 x1855007616691712/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074759.994114:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ee300 with x1855007616691712 ext(2097152->4194303) 00002000:00010000:0.0:1769074759.994120:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ee300 00000100:00100000:3.0:1769074759.995962:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616691840:12345-192.168.204.60@tcp:3:cat.0 Request processed in 48775us (51756us total) trans 0 rc 1196032/1196032 00000100:00100000:3.0:1769074759.995977:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8308 00002000:00100000:3.0:1769074759.995991:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2703360->3899391) req@ffff89b6491ec000 x1855007616691840/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:295/0 lens 488/440 e 0 to 0 dl 1769074770 ref 1 fl Complete:/600/0 rc 1196032/1196032 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074759.996001:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ec000 with x1855007616691840 ext(2703360->3899391) 00010000:00010000:3.0:1769074759.996012:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 26 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:3.0:1769074759.996031:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 26 pid: 27843 timeout: 1847 lvb_type: 0 00002000:00010000:3.0:1769074759.996040:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b6491ec000 00000100:00100000:0.0:1769074760.030476:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.030612:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616692352 00000100:00100000:1.0:1769074760.030622:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616692352 00000100:00100000:1.0:1769074760.030696:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13247 00000100:00100000:1.0:1769074760.030707:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62593:x1855007616692352:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074760.030733:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.030755:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52a71 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b195 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.030868:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.030884:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.030895:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.030922:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.030930:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.030937:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031048:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52a86 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031058:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52a86 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031064:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52a86 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074760.031073:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074760.031147:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031158:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031163:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031170:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031176:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074760.031180:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031187:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031199:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074760.031202:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074760.031207:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031212:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52a78 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031226:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031232:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031237:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031241:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074760.031244:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031249:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031256:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294a00) 00010000:00010000:1.0:1769074760.031259:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074760.031261:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031266:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52a7f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074760.031313:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52a71 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b195 expref: 34 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.031328:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52a86 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b195 expref: 33 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.031335:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074760.031337:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074760.031340:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9600, rc 0) 00000100:00100000:1.0:1769074760.031411:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+33:62593:x1855007616692352:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 706us (957us total) trans 8589943843 rc 0/0 00000100:00100000:1.0:1769074760.031420:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13247 00000100:00100000:3.0:1769074760.040087:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.040139:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.040173:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616692096 00000100:00100000:1.0:1769074760.040185:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616692096 00002000:00100000:1.0:1769074760.040254:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2703359) req@ffff89b544759500 x1855007616692096/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1769074760.040258:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625069312:192.168.204.60@tcp:104: 00002000:00100000:1.0:1769074760.040272:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759500 with x1855007616692096 ext(2097152->2703359) 00010000:00010000:1.0:1769074760.040281:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 26 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074760.040288:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.040296:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 26 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074760.040303:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00002000:00010000:1.0:1769074760.040303:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b544759500 00000100:00100000:1.0:1769074760.040332:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 691 00000100:00100000:1.0:1769074760.040341:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616692096:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074760.040357:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62590:x1855007616691968:12345-192.168.204.60@tcp:101:cat.0 Request processed in 50280us (50475us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.040365:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13706 00000100:00100000:0.0:1769074760.040384:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616692480 00000100:00100000:0.0:1769074760.040387:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616692480 00000100:00100000:0.0:1769074760.040409:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13707 00000100:00100000:0.0:1769074760.040415:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62583:x1855007616692480:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074760.040423:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.040438:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b19c expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.040455:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074760.040460:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 29 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.040469:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 29 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.040478:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718600, rc 0) 00000100:00100000:0.0:1769074760.040495:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62583:x1855007616692480:12345-192.168.204.60@tcp:101:cat.0 Request processed in 80us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.040502:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13707 00000100:00080000:3.0:1769074760.040622:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074760.045920:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.048412:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.050327:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616692736 00000100:00100000:1.0:1769074760.050335:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616692736 00000100:00100000:1.0:1769074760.050382:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4617 00000100:00100000:1.0:1769074760.050390:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62593:x1855007616692736:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074760.050666:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+33:62593:x1855007616692736:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 275us (4758us total) trans 8589943844 rc 0/0 00000100:00100000:1.0:1769074760.050673:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4617 00000100:00100000:1.0:1769074760.070506:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616692096:12345-192.168.204.60@tcp:4:cat.0 Request processed in 30154us (30421us total) trans 21474841144 rc 0/0 00000100:00100000:1.0:1769074760.070528:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 691 00002000:00100000:1.0:1769074760.070541:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2703359) req@ffff89b544759500 x1855007616692096/t21474841144(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/448 e 0 to 0 dl 1769074771 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.070551:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759500 with x1855007616692096 ext(2097152->2703359) 00010000:00010000:1.0:1769074760.070561:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 29 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074760.070579:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 29 pid: 27843 timeout: 1847 lvb_type: 0 00002000:00010000:1.0:1769074760.070586:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b544759500 00000100:00100000:3.0:1769074760.079906:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.082977:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616692992 00000100:00100000:1.0:1769074760.082994:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616692992 00010000:00010000:1.0:1769074760.083049:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 29 pid: 27843 timeout: 1847 lvb_type: 0 00000100:00100000:1.0:1769074760.083099:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6790 00000100:00100000:1.0:1769074760.083109:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2407:x1855007616692992:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074760.083126:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074760.083146:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 29 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074760.083159:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 29 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074760.083182:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd529e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->3899391] (req 2097152->2703359) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b125 expref: 29 pid: 27843 timeout: 1847 lvb_type: 0 00010000:00010000:1.0:1769074760.083197:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.083199:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.083207:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.083222:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.083230:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.083231:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.083235:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 28 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.083242:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 28 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.083275:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 28 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074760.083288:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069440:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074760.083394:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074760.083402:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069568:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074760.083421:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03180 going to sleep for 11 seconds 00010000:00010000:3.0:1769074760.093030:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074760.093242:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069568:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074760.099092:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069440:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074760.099131:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074760.099199:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616692992:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16091us (19305us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074760.099214:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6790 00000100:00100000:0.0:1769074760.104460:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.104570:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616693120 00000100:00100000:1.0:1769074760.104581:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616693120 00002000:00100000:1.0:1769074760.104638:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b546f94a80 x1855007616693120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.104648:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94a80 with x1855007616693120 ext(2097152->3145727) 00002000:00010000:1.0:1769074760.104654:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f94a80 00000100:00100000:1.0:1769074760.104692:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8309 00000100:00100000:1.0:1769074760.104701:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616693120:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074760.121423:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616693120:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16712us (16966us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074760.121447:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8309 00002000:00100000:1.0:1769074760.121460:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b546f94a80 x1855007616693120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.121471:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94a80 with x1855007616693120 ext(2097152->3145727) 00002000:00010000:1.0:1769074760.121477:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f94a80 00000100:00100000:1.0:1769074760.122631:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616693248 00000100:00100000:1.0:1769074760.122639:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616693248 00002000:00100000:1.0:1769074760.122679:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b6491eca80 x1855007616693248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.122689:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491eca80 with x1855007616693248 ext(2097152->3145727) 00002000:00010000:1.0:1769074760.122693:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491eca80 00000100:00100000:1.0:1769074760.122720:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616693376 00000100:00100000:1.0:1769074760.122724:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616693376 00002000:00100000:1.0:1769074760.122737:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b6491ed180 x1855007616693376/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.122744:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ed180 with x1855007616693376 ext(3145728->4194303) 00002000:00010000:1.0:1769074760.122747:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ed180 00000100:00100000:1.0:1769074760.122764:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8310 00000100:00100000:1.0:1769074760.122773:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616693248:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074760.123422:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8311 00000100:00100000:3.0:1769074760.123435:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616693376:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074760.144511:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.144630:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.144660:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.144692:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.144722:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616693760 00000100:00100000:0.0:1769074760.144735:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616693760 00000100:00100000:0.0:1769074760.144798:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616694016 00000100:00100000:0.0:1769074760.144801:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616694016 00000100:00100000:0.0:1769074760.144862:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616694272 00000100:00100000:0.0:1769074760.144866:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616694272 00000100:00100000:0.0:1769074760.144882:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2287 00000100:00100000:0.0:1769074760.150397:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616693760:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074760.150536:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616693760:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 5641us (5901us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.150548:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2287 00000100:00080000:3.0:1769074760.152075:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074760.152086:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074760.152090:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074760.152094:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074760.152705:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616693248:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29924us (30387us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074760.152718:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8310 00002000:00100000:3.0:1769074760.152731:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b6491eca80 x1855007616693248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.152741:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491eca80 with x1855007616693248 ext(2097152->3145727) 00002000:00010000:3.0:1769074760.152748:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491eca80 00000100:00100000:0.0:1769074760.152888:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616693376:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29456us (30540us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074760.152963:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8311 00002000:00100000:0.0:1769074760.152970:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b6491ed180 x1855007616693376/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074760.152979:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ed180 with x1855007616693376 ext(3145728->4194303) 00002000:00010000:0.0:1769074760.152983:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ed180 00000100:00100000:1.0:1769074760.153081:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616693504 00000100:00100000:1.0:1769074760.153092:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616693504 00000100:00100000:1.0:1769074760.153139:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 325 00000100:00100000:1.0:1769074760.153148:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2407:x1855007616693504:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074760.153190:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2407:x1855007616693504:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 43us (8695us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074760.153197:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 325 00000100:00100000:0.0:1769074760.158048:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.172240:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625069696:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074760.172283:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.172304:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4a00, rc 0) 00000100:00100000:0.0:1769074760.172374:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62583:x1855007616694656:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13983us (14340us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.172383:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13708 00000100:00100000:0.0:1769074760.184375:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616694784 00000100:00100000:0.0:1769074760.184390:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616694784 00002000:00100000:0.0:1769074760.184453:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3899392->4194303) req@ffff89b54d1df480 x1855007616694784/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074760.184464:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1df480 with x1855007616694784 ext(3899392->4194303) 00010000:00010000:0.0:1769074760.184473:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 19 pid: 28954 timeout: 1847 lvb_type: 0 00010000:00010000:0.0:1769074760.184486:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 19 pid: 28954 timeout: 1848 lvb_type: 0 00002000:00010000:0.0:1769074760.184494:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d1df480 00000100:00100000:0.0:1769074760.184560:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 692 00000100:00100000:0.0:1769074760.184568:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616694784:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074760.200173:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616694912 00000100:00100000:0.0:1769074760.200185:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616694912 00000100:00100000:0.0:1769074760.200248:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13709 00000100:00100000:0.0:1769074760.200258:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62590:x1855007616694912:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074760.200280:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.200300:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1aa expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.200334:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074760.200343:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 24 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.200356:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 24 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.200377:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47600, rc 0) 00000100:00100000:0.0:1769074760.200458:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62590:x1855007616694912:12345-192.168.204.60@tcp:101:cat.0 Request processed in 201us (1200us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.200466:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13709 00000100:00100000:3.0:1769074760.200606:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2408:x1855007616694784:12345-192.168.204.60@tcp:4:cat.0 Request processed in 16036us (16362us total) trans 21474841145 rc 0/0 00000100:00100000:3.0:1769074760.200614:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 692 00002000:00100000:3.0:1769074760.200623:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3899392->4194303) req@ffff89b54d1df480 x1855007616694784/t21474841145(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/448 e 0 to 0 dl 1769074771 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.200633:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1df480 with x1855007616694784 ext(3899392->4194303) 00010000:00010000:3.0:1769074760.200640:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 22 pid: 28954 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.200653:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 22 pid: 28954 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.200662:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d1df480 00000100:00100000:1.0:1769074760.219528:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695040 00000100:00100000:1.0:1769074760.219538:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695040 00010000:00010000:1.0:1769074760.219577:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 22 pid: 28954 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.219621:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6791 00000100:00100000:1.0:1769074760.219631:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616695040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074760.219648:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074760.219669:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 22 pid: 28954 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074760.219689:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 22 pid: 28954 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074760.219714:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd529ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [3899392->18446744073709551615] (req 3899392->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b12c expref: 22 pid: 28954 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074760.219730:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.219732:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.219739:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.219750:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.219757:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.219759:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.219762:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.219770:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.219803:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074760.219877:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069824:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074760.220341:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074760.220353:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069952:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074760.220379:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074760.230151:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069824:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074760.230169:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625069952:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074760.230192:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074760.230257:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616695040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10626us (10845us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074760.230267:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6791 00000100:00100000:3.0:1769074760.247066:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695168 00000100:00100000:3.0:1769074760.247076:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695168 00002000:00100000:3.0:1769074760.247125:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54d1ddc00 x1855007616695168/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.247135:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1ddc00 with x1855007616695168 ext(3145728->4194303) 00002000:00010000:3.0:1769074760.247140:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1ddc00 00000100:00100000:3.0:1769074760.247174:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8312 00000100:00100000:3.0:1769074760.247182:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616695168:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074760.266386:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695296 00000100:00100000:1.0:1769074760.266396:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695296 00002000:00100000:1.0:1769074760.266446:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54d1df800 x1855007616695296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.266459:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1df800 with x1855007616695296 ext(3145728->4194303) 00002000:00010000:1.0:1769074760.266465:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1df800 00000100:00100000:1.0:1769074760.266497:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8313 00000100:00100000:1.0:1769074760.266506:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616695296:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074760.267961:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616695168:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 20769us (21165us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074760.267973:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8312 00002000:00100000:3.0:1769074760.267985:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54d1ddc00 x1855007616695168/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.267995:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1ddc00 with x1855007616695168 ext(3145728->4194303) 00002000:00010000:3.0:1769074760.268000:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1ddc00 00000100:00100000:3.0:1769074760.277865:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695424 00000100:00100000:3.0:1769074760.277876:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695424 00002000:00100000:3.0:1769074760.277919:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54d1dd180 x1855007616695424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.277931:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dd180 with x1855007616695424 ext(4194304->4808703) 00002000:00010000:3.0:1769074760.277937:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dd180 00000100:00100000:3.0:1769074760.277970:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8314 00000100:00100000:3.0:1769074760.277979:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616695424:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074760.278288:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695552 00000100:00100000:1.0:1769074760.278292:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695552 00000100:00100000:1.0:1769074760.278322:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13710 00000100:00100000:1.0:1769074760.278329:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62595:x1855007616695552:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074760.278342:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.278373:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1b1 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.278400:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 30 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.278410:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.278419:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.278421:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.278424:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3170304->3592191] (req 3170304->3592191) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 30 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.278452:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.278462:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 22 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.278472:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070080:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074760.278556:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 30 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.278566:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 30 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.278574:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070208:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074760.278597:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074760.300025:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695680 00000100:00100000:3.0:1769074760.300037:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695680 00010000:00010000:3.0:1769074760.300079:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 31 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074760.300128:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6792 00000100:00100000:3.0:1769074760.300137:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616695680:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074760.300157:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074760.300180:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 31 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.300203:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 31 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.300241:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074760.300338:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616695680:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 202us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074760.300346:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6792 00000100:00100000:1.0:1769074760.302886:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070080:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074760.302908:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd52a9b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1a3 expref: 30 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.302931:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070208:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074760.302979:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->3592191] (req 3170304->3592191) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.302995:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9400, rc 0) 00000100:00100000:1.0:1769074760.303056:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62595:x1855007616695552:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 24726us (25210us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074760.303065:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13710 00000100:00100000:0.0:1769074760.307503:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616695424:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29512us (29748us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074760.307527:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8314 00000100:00100000:3.0:1769074760.307528:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616695296:12345-192.168.204.60@tcp:3:cat.0 Request processed in 41013us (41491us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074760.307535:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8313 00002000:00100000:0.0:1769074760.307541:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54d1dd180 x1855007616695424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.307541:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54d1df800 x1855007616695296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074760.307553:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dd180 with x1855007616695424 ext(4194304->4808703) 00002000:00100000:3.0:1769074760.307553:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1df800 with x1855007616695296 ext(3145728->4194303) 00002000:00010000:0.0:1769074760.307560:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dd180 00010000:00010000:3.0:1769074760.307563:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 21 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.307581:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52aa2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1aa expref: 21 pid: 27843 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.307589:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d1df800 00000100:00100000:0.0:1769074760.321325:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.321456:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695808 00000100:00100000:1.0:1769074760.321468:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695808 00000100:00100000:1.0:1769074760.321536:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13711 00000100:00100000:1.0:1769074760.321545:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616695808:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074760.321562:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.321579:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52ab0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1b8 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.321608:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.321620:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52ab0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b8 expref: 30 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074760.321638:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52ab0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b8 expref: 30 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074760.321656:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8e00, rc 0) 00000100:00100000:1.0:1769074760.321738:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616695808:12345-192.168.204.60@tcp:101:cat.0 Request processed in 192us (424us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074760.321746:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13711 00000100:00100000:0.0:1769074760.332138:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616695936 00000100:00100000:0.0:1769074760.332149:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616695936 00000100:00100000:0.0:1769074760.332210:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13712 00000100:00100000:0.0:1769074760.332221:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616695936:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074760.332247:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.332272:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd52ab7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1bf expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.332305:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.332326:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd52ab7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1bf expref: 22 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.332360:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd52ab7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1bf expref: 22 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.332376:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47400, rc 0) 00000100:00100000:0.0:1769074760.332454:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616695936:12345-192.168.204.60@tcp:101:cat.0 Request processed in 234us (477us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074760.332463:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13712 00000100:00100000:3.0:1769074760.336915:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.339386:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.339593:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616696192 00000100:00100000:3.0:1769074760.339601:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616696192 00000100:00100000:3.0:1769074760.339648:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13713 00000100:00100000:3.0:1769074760.339656:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616696192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074760.339672:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.339691:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52abe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1c6 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074760.339738:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070464:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074760.339832:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074760.347868:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070464:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074760.347907:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52abe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1c6 expref: 30 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.347931:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52abe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1c6 expref: 30 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.347944:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:3.0:1769074760.347991:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616696192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8336us (8610us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074760.348000:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13713 00000100:00100000:1.0:1769074760.364457:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.364524:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.364625:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616696320 00000100:00100000:0.0:1769074760.364636:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616696320 00002000:00100000:0.0:1769074760.364691:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5444a4e00 x1855007616696320/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074760.364703:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a4e00 with x1855007616696320 ext(4194304->4808703) 00002000:00010000:0.0:1769074760.364710:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a4e00 00000100:00100000:0.0:1769074760.364743:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616696448 00000100:00100000:0.0:1769074760.364747:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616696448 00002000:00100000:0.0:1769074760.364765:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3170304->3174399) req@ffff89b5444a5180 x1855007616696448/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074760.364780:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5180 with x1855007616696448 ext(3170304->3174399) 00002000:00010000:0.0:1769074760.364784:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5180 00000100:00100000:0.0:1769074760.364840:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8315 00000100:00100000:0.0:1769074760.364850:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616696320:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074760.385652:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616696320:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20791us (21194us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074760.385666:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8315 00002000:00100000:3.0:1769074760.385678:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5444a4e00 x1855007616696320/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.385690:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a4e00 with x1855007616696320 ext(4194304->4808703) 00002000:00010000:3.0:1769074760.385696:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a4e00 00000100:00100000:3.0:1769074760.395678:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616696576 00000100:00100000:3.0:1769074760.395690:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616696576 00000100:00100000:3.0:1769074760.395751:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13714 00000100:00100000:3.0:1769074760.395761:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616696576:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074760.395780:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.395799:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ac5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1cd expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074760.395908:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070592:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074760.395996:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074760.405147:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.405344:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616696704 00000100:00100000:3.0:1769074760.405357:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616696704 00002000:00100000:3.0:1769074760.405410:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3588096->3592191) req@ffff89b5444a5880 x1855007616696704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.405422:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5880 with x1855007616696704 ext(3588096->3592191) 00002000:00010000:3.0:1769074760.405429:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5880 00000100:00100000:3.0:1769074760.405468:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8317 00000100:00100000:3.0:1769074760.405478:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616696704:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074760.405675:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616696704:12345-192.168.204.60@tcp:3:cat.0 Request processed in 197us (539us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074760.405683:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8317 00002000:00100000:3.0:1769074760.405691:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3588096->3592191) req@ffff89b5444a5880 x1855007616696704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/4536 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.405700:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5880 with x1855007616696704 ext(3588096->3592191) 00002000:00010000:3.0:1769074760.405703:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5880 00002000:00010000:3.0:1769074760.407562:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074759 -> 1769074760 00000100:00100000:3.0:1769074760.407590:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070592:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074760.407629:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ac5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1cd expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.407653:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ac5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1cd expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.407667:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:3.0:1769074760.407735:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616696576:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11975us (12180us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074760.407745:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13714 00000100:00100000:3.0:1769074760.418369:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616696832 00000100:00100000:3.0:1769074760.418380:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616696832 00000100:00100000:3.0:1769074760.418440:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13715 00000100:00100000:3.0:1769074760.418449:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616696832:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074760.418467:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.418486:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52acc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1d4 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074760.418562:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070720:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074760.418651:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00002000:00010000:3.0:1769074760.428568:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074759 -> 1769074760 00002000:00010000:3.0:1769074760.428577:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074759 -> 1769074760 00000100:00100000:3.0:1769074760.428602:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070720:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074760.428642:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52acc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1d4 expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.428664:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52acc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1d4 expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.428677:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:3.0:1769074760.428744:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616696832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10295us (10481us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074760.428753:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13715 00000100:00100000:1.0:1769074760.441209:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616697088 00000100:00100000:1.0:1769074760.441220:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616697088 00000100:00100000:1.0:1769074760.441290:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4618 00000100:00100000:1.0:1769074760.441300:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+33:62590:x1855007616697088:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074760.441595:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+34:62590:x1855007616697088:12345-192.168.204.60@tcp:35:cat.0 Request processed in 296us (581us total) trans 8589943845 rc 0/0 00000100:00100000:1.0:1769074760.441604:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4618 00000100:00100000:3.0:1769074760.485905:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616697216 00000100:00100000:3.0:1769074760.485916:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616697216 00000100:00100000:3.0:1769074760.485981:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13716 00000100:00100000:3.0:1769074760.485990:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62593:x1855007616697216:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074760.486007:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.486026:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1db expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.486056:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd52975 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0df expref: 21 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.486065:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5298a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0f4 expref: 28 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.486072:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074760.486074:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074760.486079:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1249280->2097151] (req 1249280->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 21 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.486122:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5298a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0f4 expref: 28 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074760.486136:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5298a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0f4 expref: 28 pid: 27844 timeout: 1848 lvb_type: 1 00000100:00100000:3.0:1769074760.486150:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070848:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074760.486253:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd52975 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0df expref: 21 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.486269:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd52975 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0df expref: 21 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074760.486278:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070976:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074760.486301:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074760.491282:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070848:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074760.517413:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd52975 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0df expref: 21 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074760.517437:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625070976:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074760.517477:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.517492:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:3.0:1769074760.519384:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62593:x1855007616697216:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 33383us (34076us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074760.519396:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13716 00000100:00100000:3.0:1769074760.519738:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625071104:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074760.519771:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074760.520032:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616697344:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10454us (12260us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074760.520042:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6795 00000100:00100000:0.0:1769074760.530107:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.533866:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616697728 00000100:00100000:3.0:1769074760.533877:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616697728 00000100:00100000:3.0:1769074760.533947:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616697856 00000100:00100000:3.0:1769074760.533950:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616697856 00000100:00100000:3.0:1769074760.533970:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13717 00000100:00100000:3.0:1769074760.533978:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616697728:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074760.533998:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.534019:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52ada lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1e2 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074760.534076:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13718 00000100:00100000:0.0:1769074760.534085:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616697856:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:3.0:1769074760.534090:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625071232:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074760.534092:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.534100:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd52ae1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1e9 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074760.534128:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625071360:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074760.534496:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616697600 00000100:00100000:1.0:1769074760.534503:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616697600 00002000:00100000:1.0:1769074760.534545:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1249280->1253375) req@ffff89b542fa8700 x1855007616697600/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.534555:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8700 with x1855007616697600 ext(1249280->1253375) 00002000:00010000:1.0:1769074760.534560:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b542fa8700 00000100:00100000:1.0:1769074760.534575:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8318 00000100:00100000:1.0:1769074760.534582:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616697600:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074760.537160:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074760.540128:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616697600:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 5534us (10018us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074760.540142:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8318 00002000:00100000:1.0:1769074760.540154:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1249280->1253375) req@ffff89b542fa8700 x1855007616697600/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/4536 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.540171:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8700 with x1855007616697600 ext(1249280->1253375) 00002000:00010000:1.0:1769074760.540177:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b542fa8700 00000100:00100000:0.0:1769074760.543175:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625071360:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074760.543226:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd52ae1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1e9 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.543253:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd52ae1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1e9 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.543269:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47e00, rc 0) 00000100:00100000:0.0:1769074760.543354:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616697856:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9269us (9690us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074760.543369:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13718 00002000:00010000:1.0:1769074760.544140:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074758 -> 1769074760 00002000:00010000:1.0:1769074760.544145:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074757 -> 1769074760 00002000:00010000:1.0:1769074760.544147:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074758 -> 1769074760 00000100:00100000:1.0:1769074760.544160:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625071232:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074760.544181:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52ada lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1e2 expref: 29 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074760.544207:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52ada lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1e2 expref: 29 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074760.544217:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:1.0:1769074760.544259:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616697728:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10281us (10648us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074760.544265:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13717 00000100:00100000:1.0:1769074760.576523:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616697984 00000100:00100000:1.0:1769074760.576536:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616697984 00000100:00100000:1.0:1769074760.576600:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13719 00000100:00100000:1.0:1769074760.576611:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62601:x1855007616697984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074760.576632:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.576662:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1f0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.576703:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.576713:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.576716:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.576720:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.576766:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.576782:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.576799:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625071488:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074760.590616:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616698112 00000100:00100000:0.0:1769074760.590632:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616698112 00000100:00100000:0.0:1769074760.590698:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13720 00000100:00100000:0.0:1769074760.590708:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62593:x1855007616698112:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074760.590729:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.590751:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1f7 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.590785:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52913 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0a7 expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.590795:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52944 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0c3 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.590803:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.590837:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.590843:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->1138687] (req 1048576->1138687) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.590888:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52944 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0c3 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.590909:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52944 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0c3 expref: 22 pid: 27844 timeout: 1848 lvb_type: 1 00000100:00100000:0.0:1769074760.590923:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625071616:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074760.591025:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52913 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0a7 expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.591036:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71a00/0x7fe2323adfd52913 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0a7 expref: 28 pid: 28954 timeout: 1848 lvb_type: 0 00000100:00100000:0.0:1769074760.591044:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625071744:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074760.591069:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074760.597434:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.597544:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616698240 00000100:00100000:3.0:1769074760.597557:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616698240 00002000:00100000:3.0:1769074760.597623:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1249280->2097151) req@ffff89b546f90700 x1855007616698240/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.597634:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90700 with x1855007616698240 ext(1249280->2097151) 00010000:00010000:3.0:1769074760.597646:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 20 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.597661:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 20 pid: 27844 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.597669:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f90700 00000100:00100000:3.0:1769074760.597710:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 693 00000100:00100000:3.0:1769074760.597719:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616698240:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074760.607319:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625071616:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074760.608724:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.610282:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616698496 00000100:00100000:3.0:1769074760.610300:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616698496 00000100:00100000:3.0:1769074760.610522:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13248 00000100:00100000:3.0:1769074760.610532:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+34:62606:x1855007616698496:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074760.610558:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.610582:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52af6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b1fe expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.610669:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.610685:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.610699:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.610714:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.610721:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.610727:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611318:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52b0b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611333:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52b0b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611341:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52b0b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074760.611353:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074760.611448:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611455:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611462:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611469:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611475:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074760.611480:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611487:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611498:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074760.611501:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074760.611504:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611528:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52afd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611542:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611548:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611554:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611563:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ea00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074760.611565:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611571:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611577:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ea700) 00010000:00010000:3.0:1769074760.611580:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074760.611582:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611587:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b04 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.611627:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52af6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1fe expref: 37 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.611639:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52b0b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1fe expref: 36 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.611647:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074760.611649:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074760.611652:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e800, rc 0) 00000100:00100000:3.0:1769074760.611728:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+36:62606:x1855007616698496:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1198us (3013us total) trans 8589943846 rc 0/0 00000100:00100000:3.0:1769074760.611736:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13248 00000100:00100000:0.0:1769074760.618095:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625071744:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074760.618163:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->1138687] (req 1048576->1138687) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.618190:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47e00, rc 0) 00000100:00100000:0.0:1769074760.618265:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62593:x1855007616698112:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 27557us (28479us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.618275:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13720 00000100:00100000:1.0:1769074760.627352:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616698880 00000100:00100000:1.0:1769074760.627361:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616698880 00000100:00100000:1.0:1769074760.627409:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616699008 00000100:00100000:1.0:1769074760.627413:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616699008 00000100:00100000:1.0:1769074760.627433:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13721 00000100:00100000:1.0:1769074760.627442:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616698880:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074760.627462:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.627479:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52b19 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b205 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074760.627547:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625071872:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074760.627621:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a8b3cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074760.627692:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13722 00000100:00100000:0.0:1769074760.627701:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b35b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616699008:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074760.627710:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.627720:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd52b20 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b20c expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.627734:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.627745:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd52b20 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b20c expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.627764:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd52b20 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b20c expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074760.627777:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47600, rc 0) 00000100:00100000:0.0:1769074760.627870:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b35b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616699008:12345-192.168.204.60@tcp:101:cat.0 Request processed in 170us (717us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074760.627878:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13722 00000100:00100000:0.0:1769074760.636859:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616699136 00000100:00100000:0.0:1769074760.636876:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616699136 00000100:00100000:0.0:1769074760.636933:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13723 00000100:00100000:0.0:1769074760.636942:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2406:x1855007616699136:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074760.636961:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074760.636981:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47600/0x7fe2323adfd52b27 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b213 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074760.637046:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625072000:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074760.637152:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074760.639800:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.639932:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.640029:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625071872:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074760.640064:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52b19 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b205 expref: 25 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074760.640091:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52b19 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b205 expref: 25 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074760.640103:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8e00, rc 0) 00000100:00100000:1.0:1769074760.640151:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2406:x1855007616698880:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12710us (13028us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074760.640160:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13721 00000100:00100000:3.0:1769074760.658495:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497abb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625072128:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074760.658535:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074760.658599:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616699264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17490us (18665us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074760.658613:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6797 00000100:00100000:0.0:1769074760.670723:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.670853:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.670909:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.672700:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616699776 00000100:00100000:1.0:1769074760.672706:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616699776 00000100:00100000:1.0:1769074760.672749:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13724 00000100:00100000:1.0:1769074760.672756:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62606:x1855007616699776:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074760.672768:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.672790:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b21a expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.672848:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.672855:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.672868:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.672880:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9a00, rc 0) 00000100:00100000:1.0:1769074760.672912:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62606:x1855007616699776:12345-192.168.204.60@tcp:101:cat.0 Request processed in 156us (2002us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074760.672918:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13724 00000100:00100000:3.0:1769074760.682424:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616699392 00000100:00100000:3.0:1769074760.682444:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616699392 00010000:00010000:0.0:1769074760.683044:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 11 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:0.0:1769074760.683101:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6798 00000100:00100000:0.0:1769074760.683121:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616699392:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074760.683141:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074760.683167:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 11 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074760.683183:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 11 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074760.683209:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52ad3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1249280->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1db expref: 11 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074760.683228:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.683230:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.683241:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.683253:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.683263:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.683265:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074760.683268:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.683275:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.683320:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074760.683354:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542faa300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625072256:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074760.683789:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074760.683799:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625072384:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074760.683894:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984240 going to sleep for 11 seconds 00000100:00100000:0.0:1769074760.693864:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542faa300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625072256:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074760.719682:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.719791:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616699904 00000100:00100000:3.0:1769074760.719851:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616699904 00002000:00100000:3.0:1769074760.720173:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b542fa9500 x1855007616699904/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.720183:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9500 with x1855007616699904 ext(1048576->2097151) 00002000:00010000:3.0:1769074760.720188:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b542fa9500 00000100:00100000:3.0:1769074760.720225:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8320 00000100:00100000:3.0:1769074760.720234:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616699904:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074760.737929:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.737988:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.739045:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8322 00000100:00100000:1.0:1769074760.739055:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616700160:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074760.750679:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616699904:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30434us (30998us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074760.750693:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8320 00002000:00100000:1.0:1769074760.750705:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b542fa9500 x1855007616699904/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074760.750716:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9500 with x1855007616699904 ext(1048576->2097151) 00002000:00010000:1.0:1769074760.750722:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b542fa9500 00000100:00100000:3.0:1769074760.798084:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616700032:12345-192.168.204.60@tcp:3:cat.0 Request processed in 59453us (60167us total) trans 0 rc 958464/958464 00000100:00100000:3.0:1769074760.798096:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8321 00002000:00100000:3.0:1769074760.798109:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1138688->2097151) req@ffff89b542fa8e00 x1855007616700032/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/440 e 0 to 0 dl 1769074771 ref 1 fl Complete:/600/0 rc 958464/958464 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.798119:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8e00 with x1855007616700032 ext(1138688->2097151) 00002000:00010000:3.0:1769074760.798125:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa8e00 00000100:00100000:1.0:1769074760.798258:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616700288 00000100:00100000:1.0:1769074760.798266:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616700288 00000100:00100000:1.0:1769074760.798496:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13725 00000100:00100000:1.0:1769074760.798532:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62601:x1855007616700288:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074760.798548:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.798567:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b221 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.798596:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.798609:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.798611:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.798615:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.798650:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.798663:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 20 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.798676:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625072512:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074760.798742:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a8b3cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074760.819345:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625072512:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074760.819382:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.819401:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:1.0:1769074760.819469:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62601:x1855007616700288:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20964us (21727us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074760.819477:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13725 00000100:00100000:0.0:1769074760.829267:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.829635:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616700416 00000100:00100000:3.0:1769074760.829649:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616700416 00002000:00100000:3.0:1769074760.829710:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1138687) req@ffff89b542fa8000 x1855007616700416/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.829721:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8000 with x1855007616700416 ext(1048576->1138687) 00010000:00010000:3.0:1769074760.829732:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 21 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.829762:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 21 pid: 27844 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.829770:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b542fa8000 00000100:00100000:3.0:1769074760.829849:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 694 00000100:00100000:3.0:1769074760.829860:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616700416:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074760.843882:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2406:x1855007616700416:12345-192.168.204.60@tcp:4:cat.0 Request processed in 14012us (14614us total) trans 17179874468 rc 0/0 00000100:00100000:3.0:1769074760.843897:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 694 00002000:00100000:3.0:1769074760.843911:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1138687) req@ffff89b542fa8000 x1855007616700416/t17179874468(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/448 e 0 to 0 dl 1769074771 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.843921:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8000 with x1855007616700416 ext(1048576->1138687) 00010000:00010000:3.0:1769074760.843931:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 23 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.843950:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47e00/0x7fe2323adfd52aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1138687) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f7 expref: 23 pid: 27844 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.843959:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b542fa8000 00000001:00080000:0.0:1769074760.851308:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874468 is committed 00000001:00080000:0.0:1769074760.851328:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874467 is committed 00000001:00080000:0.0:1769074760.851366:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874457 is committed 00000001:00080000:0.0:1769074760.851405:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874430 is committed 00000100:00100000:0.0:1769074760.853143:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.855781:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616700544 00000100:00100000:1.0:1769074760.855798:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616700544 00000100:00100000:1.0:1769074760.855933:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13726 00000100:00100000:1.0:1769074760.855943:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62606:x1855007616700544:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074760.855961:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074760.855979:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b228 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.856011:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.856019:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.856022:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074760.856026:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.856062:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.856074:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 16 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.856088:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625072640:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074760.856183:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a8b3cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074760.881778:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074760.881879:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.882146:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616700672 00000100:00100000:3.0:1769074760.882159:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616700672 00002000:00100000:3.0:1769074760.882214:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3170304->3592191) req@ffff89b542fabb80 x1855007616700672/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.882231:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fabb80 with x1855007616700672 ext(3170304->3592191) 00010000:00010000:3.0:1769074760.882240:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 17 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.882254:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 17 pid: 27844 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.882262:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b542fabb80 00000100:00100000:3.0:1769074760.882303:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 695 00000100:00100000:3.0:1769074760.882312:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616700672:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074760.889499:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074760.890465:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616701056 00000100:00100000:3.0:1769074760.890477:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616701056 00000100:00100000:3.0:1769074760.890548:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13249 00000100:00100000:3.0:1769074760.890558:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+35:62609:x1855007616701056:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074760.890584:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074760.890606:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52b43 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b22f expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.890697:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.890714:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.890725:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.890743:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.890750:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.890756:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891229:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52b58 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891242:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52b58 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891249:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52b58 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074760.891258:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074760.891338:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891345:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891352:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891358:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891364:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ea00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074760.891368:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891375:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891387:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074760.891390:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074760.891395:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891401:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52b4a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891414:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891419:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891424:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891429:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074760.891431:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891436:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891444:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ea700) 00010000:00010000:3.0:1769074760.891446:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074760.891449:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891455:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52b51 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074760.891502:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd52b43 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b22f expref: 38 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.891526:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52b58 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b22f expref: 37 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074760.891534:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074760.891536:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074760.891539:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:3.0:1769074760.891874:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+37:62609:x1855007616701056:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1316us (2385us total) trans 8589943847 rc 0/0 00000100:00100000:3.0:1769074760.891883:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13249 00000100:00100000:3.0:1769074760.902124:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616700672:12345-192.168.204.60@tcp:4:cat.0 Request processed in 19801us (20344us total) trans 21474841147 rc 0/0 00000100:00100000:3.0:1769074760.902140:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 695 00002000:00100000:3.0:1769074760.902152:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3170304->3592191) req@ffff89b542fabb80 x1855007616700672/t21474841147(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:296/0 lens 488/448 e 0 to 0 dl 1769074771 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074760.902163:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fabb80 with x1855007616700672 ext(3170304->3592191) 00010000:00010000:3.0:1769074760.902173:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074760.902191:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074760.902199:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b542fabb80 00000100:00100000:1.0:1769074760.918206:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.918313:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.918352:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.918388:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074760.918417:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616701312 00000100:00100000:1.0:1769074760.918428:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616701312 00000100:00100000:1.0:1769074760.918488:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616701696 00000100:00100000:1.0:1769074760.918492:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616701696 00010000:00010000:1.0:1769074760.918518:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074760.918545:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6800 00000100:00100000:1.0:1769074760.918554:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616701696:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074760.918580:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074760.918602:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074760.918617:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074760.918643:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52aa9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3170304->18446744073709551615] (req 3170304->3592191) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1b1 expref: 19 pid: 27844 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074760.918660:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.918662:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074760.918670:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.918682:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074760.918719:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074760.918739:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625072896:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074760.918865:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a8b3cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074760.919266:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616701568 00000100:00100000:0.0:1769074760.919272:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616701568 00000100:00100000:0.0:1769074760.919316:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4619 00000100:00100000:0.0:1769074760.919324:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e9c0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+38:62609:x1855007616701568:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074760.919526:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e9c0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+39:62609:x1855007616701568:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 202us (1172us total) trans 8589943848 rc 0/0 00000100:00100000:0.0:1769074760.919537:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4619 00000100:00100000:0.0:1769074760.919595:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 983 00000100:00100000:0.0:1769074760.919603:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+38:2408:x1855007616701312:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074760.919611:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074760.919629:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52b58 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b22f expref: 38 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074760.919642:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074760.919647:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074760.919664:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+37:2408:x1855007616701312:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 63us (1352us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074760.919670:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 983 00000100:00100000:3.0:1769074760.921449:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616701184 00000100:00100000:3.0:1769074760.921466:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616701184 00000100:00100000:1.0:1769074760.932394:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625072896:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074760.932423:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074760.932521:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2406:x1855007616701696:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13967us (14131us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074760.932533:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6800 00000100:00100000:0.0:1769074760.954286:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616701824 00000100:00100000:0.0:1769074760.954299:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616701824 00002000:00100000:0.0:1769074760.954378:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b5497a9180 x1855007616701824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:296/0 lens 488/0 e 0 to 0 dl 1769074771 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074760.954391:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9180 with x1855007616701824 ext(3145728->4194303) 00002000:00010000:0.0:1769074760.954398:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497a9180 00000100:00100000:0.0:1769074760.954520:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8324 00000100:00100000:0.0:1769074760.954535:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616701824:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074760.980640:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.008403:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.012960:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702208 00000100:00100000:3.0:1769074761.012973:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702208 00000100:00100000:3.0:1769074761.013024:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702336 00000100:00100000:3.0:1769074761.013027:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702336 00000100:00100000:3.0:1769074761.016734:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13728 00000100:00100000:3.0:1769074761.016755:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2406:x1855007616702208:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.016777:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.016799:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b23d expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018274:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.018278:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.018299:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 9 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.018336:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 9 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.018360:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba400, rc 0) 00000100:00100000:3.0:1769074761.018474:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616702208:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1723us (6078us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.018486:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13728 00000100:00100000:3.0:1769074761.018544:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13729 00000100:00100000:3.0:1769074761.018567:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62609:x1855007616702336:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074761.018588:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.018611:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b244 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018641:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018654:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52b66 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b236 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018663:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018671:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018678:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:3.0:1769074761.018681:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:3.0:1769074761.018685:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 7/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2953216->3145727] (req 2953216->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018733:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.018752:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 20 pid: 28955 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.018769:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073024:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.019440:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.019452:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 12 pid: 28955 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.019462:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073152:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.030778:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52b66 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b236 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.030798:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52b66 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b236 expref: 20 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.030839:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073280:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.030899:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.030908:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 12 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.030916:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073408:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.030934:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.031331:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.031371:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702464 00000100:00100000:1.0:1769074761.031380:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702464 00010000:00010000:1.0:1769074761.031416:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 21 pid: 28955 timeout: 1848 lvb_type: 0 00000100:00100000:1.0:1769074761.031450:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6801 00000100:00100000:1.0:1769074761.031458:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616702464:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074761.031472:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074761.031492:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 21 pid: 28955 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074761.031517:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 21 pid: 28955 timeout: 1848 lvb_type: 0 00010000:00010000:1.0:1769074761.031550:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074761.031587:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616702464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 125us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.031599:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6801 00010000:00010000:3.0:1769074761.032462:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52a94 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b19c expref: 20 pid: 28955 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.032479:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073024:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.032613:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616702080:12345-192.168.204.60@tcp:3:cat.0 Request processed in 23672us (24220us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074761.032621:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8325 00002000:00100000:3.0:1769074761.032634:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54475a300 x1855007616702080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.032643:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475a300 with x1855007616702080 ext(3145728->4194303) 00010000:00010000:3.0:1769074761.032652:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52b66 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b236 expref: 18 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:3.0:1769074761.032664:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd52b66 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b236 expref: 18 pid: 27843 timeout: 1848 lvb_type: 0 00002000:00010000:3.0:1769074761.032671:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54475a300 00000100:00100000:3.0:1769074761.042526:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073152:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.042557:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073280:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.042575:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073408:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.042637:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2953216->3145727] (req 2953216->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.042664:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb000, rc 0) 00000100:00100000:3.0:1769074761.042750:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62609:x1855007616702336:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 24192us (30195us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.042760:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13729 00000100:00100000:0.0:1769074761.054205:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.054265:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.054304:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.054347:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702720 00000100:00100000:0.0:1769074761.054359:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702720 00010000:00010000:0.0:1769074761.054405:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 12 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:0.0:1769074761.054443:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702592 00000100:00100000:0.0:1769074761.054446:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702592 00010000:00010000:0.0:1769074761.054460:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 13 pid: 28955 timeout: 1848 lvb_type: 0 00000100:00100000:0.0:1769074761.054479:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6802 00000100:00100000:0.0:1769074761.054487:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616702720:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.054514:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.054537:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 13 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.054549:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 13 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.054571:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52b3c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b228 expref: 13 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.054587:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.054639:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616702720:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 151us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.054647:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6802 00000100:00100000:0.0:1769074761.054698:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6803 00000100:00100000:0.0:1769074761.054704:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616702592:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.054712:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.054721:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 11 pid: 28955 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.054732:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 11 pid: 28955 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.054744:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52a6a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b18e expref: 11 pid: 28955 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.054754:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.054770:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616702592:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 66us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.054776:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6803 00000100:00100000:3.0:1769074761.055366:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702848 00000100:00100000:3.0:1769074761.055375:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702848 00000100:00100000:3.0:1769074761.055420:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13730 00000100:00100000:3.0:1769074761.055428:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616702848:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.055443:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.055459:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52b7b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b24b expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.055476:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.055486:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52b7b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b24b expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.055500:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52b7b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b24b expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.055521:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074761.055564:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616702848:12345-192.168.204.60@tcp:101:cat.0 Request processed in 138us (1299us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074761.055570:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13730 00000100:00100000:0.0:1769074761.065337:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.065452:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.066501:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616702976 00000100:00100000:1.0:1769074761.066516:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616702976 00000100:00100000:1.0:1769074761.066557:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13731 00000100:00100000:1.0:1769074761.066565:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616702976:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.066577:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.066589:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd52b82 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b252 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074761.066622:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625073664:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074761.066666:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a8b3cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074761.074097:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.075049:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616703232 00000100:00100000:3.0:1769074761.075059:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616703232 00000100:00100000:3.0:1769074761.075115:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13732 00000100:00100000:3.0:1769074761.075125:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616703232:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.075141:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.075159:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52b89 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b259 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074761.075216:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073792:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074761.075294:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.078154:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625073664:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074761.078219:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd52b82 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b252 expref: 18 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074761.078248:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd52b82 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b252 expref: 18 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074761.078262:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:1.0:1769074761.078369:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616702976:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11803us (13042us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074761.078378:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13731 00000100:00100000:0.0:1769074761.091470:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.091528:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.091655:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073792:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074761.091691:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52b89 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b259 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.091714:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52b89 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b259 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.091726:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074761.091785:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616703232:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16661us (17698us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074761.091794:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13732 00000100:00100000:3.0:1769074761.091882:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616703488 00000100:00100000:3.0:1769074761.091888:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616703488 00000100:00100000:3.0:1769074761.091926:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13733 00000100:00100000:3.0:1769074761.091934:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616703488:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.091948:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.091963:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52b90 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b260 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074761.092013:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073920:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074761.092064:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074761.108460:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.108537:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.108669:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625073920:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074761.108704:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52b90 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b260 expref: 18 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.108729:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52b90 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b260 expref: 18 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.108740:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:3.0:1769074761.108799:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616703488:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16864us (17270us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074761.108862:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13733 00000100:00100000:3.0:1769074761.108889:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616703872 00000100:00100000:3.0:1769074761.108896:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616703872 00000100:00100000:3.0:1769074761.108934:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13734 00000100:00100000:3.0:1769074761.108942:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62609:x1855007616703872:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074761.108957:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.108972:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b267 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.109004:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.109013:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.109021:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.109024:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.109028:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3346431] (req 3145728->3346431) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.109072:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.109088:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd52a63 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b187 expref: 10 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.109106:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625074048:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.109169:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.109179:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 20 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:3.0:1769074761.109187:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625074176:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.109210:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.110120:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616703744 00000100:00100000:1.0:1769074761.110132:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616703744 00000100:00100000:1.0:1769074761.110192:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4620 00000100:00100000:1.0:1769074761.110201:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+37:62606:x1855007616703744:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074761.110396:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+38:62606:x1855007616703744:12345-192.168.204.60@tcp:35:cat.0 Request processed in 195us (1944us total) trans 8589943849 rc 0/0 00000100:00100000:1.0:1769074761.110402:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4620 00000100:00080000:0.0:1769074761.111117:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074761.115427:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625074048:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.115443:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625074176:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.115475:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3346431] (req 3145728->3346431) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.115493:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074761.115587:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62609:x1855007616703872:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6645us (7047us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.115600:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13734 00000100:00100000:0.0:1769074761.134185:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.134232:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.134315:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704000 00000100:00100000:3.0:1769074761.134336:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704000 00000100:00100000:3.0:1769074761.134400:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13735 00000100:00100000:3.0:1769074761.134409:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616704000:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.134428:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.134445:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52b9e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b26e expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074761.134500:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074304:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074761.134593:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.159770:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074304:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074761.159859:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52b9e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b26e expref: 18 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.159884:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52b9e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b26e expref: 18 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.159897:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:3.0:1769074761.159957:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616704000:12345-192.168.204.60@tcp:101:cat.0 Request processed in 25549us (25783us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074761.159968:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13735 00000100:00100000:1.0:1769074761.186197:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.186298:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704256 00000100:00100000:0.0:1769074761.186308:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704256 00010000:00010000:0.0:1769074761.186368:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 21 pid: 27843 timeout: 1848 lvb_type: 0 00000100:00100000:0.0:1769074761.186417:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6806 00000100:00100000:0.0:1769074761.186427:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616704256:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.186451:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.186473:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 21 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.186487:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 21 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.186521:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd52a5c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b180 expref: 21 pid: 27843 timeout: 1848 lvb_type: 0 00010000:00010000:0.0:1769074761.186538:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074761.186541:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074761.186548:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.186558:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.186596:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074761.186609:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625074432:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074761.186725:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074761.197693:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625074432:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074761.197721:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.197785:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616704256:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11358us (11597us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.197797:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6806 00000100:00100000:3.0:1769074761.209197:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.209584:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704384 00000100:00100000:1.0:1769074761.209596:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704384 00002000:00100000:1.0:1769074761.209647:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3342336->3346431) req@ffff89b544759180 x1855007616704384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.209659:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759180 with x1855007616704384 ext(3342336->3346431) 00002000:00010000:1.0:1769074761.209664:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b544759180 00000100:00100000:1.0:1769074761.209706:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8327 00000100:00100000:1.0:1769074761.209715:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616704384:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074761.209977:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616704384:12345-192.168.204.60@tcp:3:cat.0 Request processed in 262us (792us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074761.209986:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8327 00002000:00100000:1.0:1769074761.209994:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3342336->3346431) req@ffff89b544759180 x1855007616704384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/4536 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.210002:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759180 with x1855007616704384 ext(3342336->3346431) 00002000:00010000:1.0:1769074761.210006:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b544759180 00000100:00100000:1.0:1769074761.218983:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.219091:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704512 00000100:00100000:3.0:1769074761.219107:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704512 00000100:00100000:3.0:1769074761.219194:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13736 00000100:00100000:3.0:1769074761.219206:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62603:x1855007616704512:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074761.219227:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.219248:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b275 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.219289:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.219299:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.219306:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.219308:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.219312:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1593344->2097151] (req 1593344->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.219362:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.219378:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 9 pid: 27844 timeout: 1849 lvb_type: 1 00000100:00100000:3.0:1769074761.219394:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074560:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.219562:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.219578:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.219589:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074688:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.219618:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.225370:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074560:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.225384:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074688:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.225414:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1593344->2097151] (req 1593344->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.225432:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:3.0:1769074761.225495:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62603:x1855007616704512:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6291us (6522us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.225515:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13736 00000100:00100000:1.0:1769074761.243147:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.243193:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.244899:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704768 00000100:00100000:0.0:1769074761.244914:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704768 00010000:00010000:0.0:1769074761.244971:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.245013:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704640 00000100:00100000:0.0:1769074761.245016:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704640 00010000:00010000:0.0:1769074761.245029:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 10 pid: 27844 timeout: 1849 lvb_type: 1 00000100:00100000:0.0:1769074761.245056:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6807 00000100:00100000:0.0:1769074761.245065:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2406:x1855007616704768:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.245079:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.245101:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.245113:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.245140:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52a0f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b14f expref: 18 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.245164:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.245226:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616704768:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 162us (2090us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.245234:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6807 00000100:00100000:0.0:1769074761.245251:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6808 00000100:00100000:0.0:1769074761.245257:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616704640:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.245264:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.245272:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 10 pid: 27844 timeout: 1849 lvb_type: 1 00010000:00010000:0.0:1769074761.245284:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 10 pid: 27844 timeout: 1849 lvb_type: 1 00010000:00010000:0.0:1769074761.245296:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52a47 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b16b expref: 10 pid: 27844 timeout: 1849 lvb_type: 1 00010000:00010000:0.0:1769074761.245305:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074761.245307:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074761.245313:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.245324:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.245362:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074761.245376:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625074816:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074761.245429:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.253977:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.254139:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616704896 00000100:00100000:3.0:1769074761.254154:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616704896 00002000:00100000:3.0:1769074761.254217:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1593344->1597439) req@ffff89b5462de300 x1855007616704896/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.254229:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462de300 with x1855007616704896 ext(1593344->1597439) 00002000:00010000:3.0:1769074761.254237:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462de300 00000100:00100000:3.0:1769074761.254282:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8328 00000100:00100000:3.0:1769074761.254292:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616704896:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074761.254525:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616704896:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 231us (559us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074761.254533:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8328 00002000:00100000:3.0:1769074761.254539:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1593344->1597439) req@ffff89b5462de300 x1855007616704896/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/4536 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.254547:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462de300 with x1855007616704896 ext(1593344->1597439) 00002000:00010000:3.0:1769074761.254551:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462de300 00000100:00100000:0.0:1769074761.263028:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625074816:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074761.263054:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.263137:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616704640:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17871us (19935us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.263149:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6808 00000100:00100000:1.0:1769074761.270749:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.270855:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616705024 00000100:00100000:3.0:1769074761.270866:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616705024 00000100:00100000:3.0:1769074761.270982:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13737 00000100:00100000:3.0:1769074761.270994:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62603:x1855007616705024:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074761.271021:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.271047:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b27c expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.271083:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.271093:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.271100:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.271102:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.271107:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1589247] (req 1048576->1589247) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.271156:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.271168:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52b2e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b21a expref: 8 pid: 28955 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.271182:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074944:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.271312:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.274209:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd52ae8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b1f0 expref: 18 pid: 27844 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.274231:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625075072:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.274335:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.283763:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625074944:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.283779:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625075072:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.284038:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1589247] (req 1048576->1589247) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.284060:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640030a00, rc 0) 00000100:00100000:3.0:1769074761.284163:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62603:x1855007616705024:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 13171us (13423us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.284176:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13737 00000100:00100000:0.0:1769074761.294660:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.294754:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.294795:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.296985:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616705280 00000100:00100000:3.0:1769074761.297000:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616705280 00000100:00100000:0.0:1769074761.305528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.305661:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616705664 00000100:00100000:3.0:1769074761.305673:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616705664 00002000:00100000:3.0:1769074761.305733:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1585152->1589247) req@ffff89b5499d2a00 x1855007616705664/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.305746:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2a00 with x1855007616705664 ext(1585152->1589247) 00002000:00010000:3.0:1769074761.305752:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5499d2a00 00000100:00100000:3.0:1769074761.305790:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8329 00000100:00100000:3.0:1769074761.305800:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616705664:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074761.306003:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616705664:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 203us (484us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074761.306011:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8329 00002000:00100000:3.0:1769074761.306018:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1585152->1589247) req@ffff89b5499d2a00 x1855007616705664/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/4536 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.306026:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2a00 with x1855007616705664 ext(1585152->1589247) 00002000:00010000:3.0:1769074761.306030:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5499d2a00 00000100:00100000:1.0:1769074761.307209:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625075200:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074761.307233:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074761.307295:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2406:x1855007616705408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11940us (12499us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.307304:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6810 00000100:00100000:0.0:1769074761.317925:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.318632:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616705792 00000100:00100000:1.0:1769074761.318646:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616705792 00000100:00100000:1.0:1769074761.318730:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 984 00000100:00100000:1.0:1769074761.318739:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+40:2407:x1855007616705792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074761.318759:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074761.318788:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd52bc8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b283 expref: 40 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.318840:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074761.319055:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074761.319135:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+39:2407:x1855007616705792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 396us (1221us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.319144:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 984 00000100:00100000:0.0:1769074761.322360:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.322422:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.322509:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616705920 00000100:00100000:1.0:1769074761.322519:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616705920 00000100:00100000:1.0:1769074761.322576:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616706048 00000100:00100000:1.0:1769074761.322579:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616706048 00000100:00100000:1.0:1769074761.322604:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13738 00000100:00100000:1.0:1769074761.322613:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616705920:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.322632:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.322651:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52bd6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b28a expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074761.322710:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075328:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074761.322783:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.322800:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13739 00000100:00100000:1.0:1769074761.322835:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616706048:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.322842:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.322859:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52bdd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b291 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074761.322885:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625075456:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074761.322912:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.325333:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625075456:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074761.325369:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52bdd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b291 expref: 11 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074761.325391:0:28519:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52bdd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b291 expref: 11 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074761.325409:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867c00, rc 0) 00000100:00100000:1.0:1769074761.325468:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616706048:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2660us (3046us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074761.325477:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13739 00002000:00010000:0.0:1769074761.334552:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074759 -> 1769074760 00002000:00010000:0.0:1769074761.334561:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074759 -> 1769074760 00002000:00010000:0.0:1769074761.334563:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074759 -> 1769074760 00000100:00100000:0.0:1769074761.334587:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075328:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074761.334627:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52bd6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b28a expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074761.334649:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52bd6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b28a expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074761.334666:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:0.0:1769074761.334724:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616705920:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12112us (12375us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074761.334734:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13738 00000100:00100000:0.0:1769074761.347687:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.347798:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616706176 00000100:00100000:1.0:1769074761.347842:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616706176 00000100:00100000:1.0:1769074761.347906:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13740 00000100:00100000:1.0:1769074761.347915:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+8:62615:x1855007616706176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.347937:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.347956:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b298 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.347988:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.347997:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074761.347999:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074761.348003:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.348038:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 16 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.348050:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 16 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:1.0:1769074761.348063:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075584:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074761.348137:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.359649:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075584:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074761.359684:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.359703:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867c00, rc 0) 00000100:00100000:1.0:1769074761.359784:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62615:x1855007616706176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11868us (12108us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.359793:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13740 00000100:00100000:0.0:1769074761.363599:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.363719:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616706432 00000100:00100000:1.0:1769074761.363733:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616706432 00000100:00100000:1.0:1769074761.363803:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13741 00000100:00100000:1.0:1769074761.363849:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616706432:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.363870:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.363891:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52beb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b29f expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074761.363952:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075712:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074761.364042:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074761.366374:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.366415:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.367217:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616706304 00000100:00100000:3.0:1769074761.367222:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616706304 00002000:00100000:3.0:1769074761.367251:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1593344->2097151) req@ffff89b5499d1180 x1855007616706304/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.367258:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1180 with x1855007616706304 ext(1593344->2097151) 00010000:00010000:3.0:1769074761.367265:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074761.367276:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:3.0:1769074761.367281:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5499d1180 00000100:00100000:3.0:1769074761.367301:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 696 00000100:00100000:3.0:1769074761.367305:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616706304:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074761.367828:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075712:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074761.367859:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52beb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b29f expref: 20 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074761.367877:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52beb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b29f expref: 20 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074761.367888:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:1.0:1769074761.367921:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616706432:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4075us (4334us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074761.367929:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13741 00000100:00100000:0.0:1769074761.368926:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000001:00080000:0.0:1769074761.371515:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943851 is committed 00000001:00080000:0.0:1769074761.371553:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943838 is committed 00000001:00080000:0.0:1769074761.371561:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943833 is committed 00000100:00100000:0.0:1769074761.372796:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.376140:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616706688 00000100:00100000:1.0:1769074761.376158:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616706688 00000100:00100000:1.0:1769074761.376226:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13743 00000100:00100000:1.0:1769074761.376234:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616706688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.376251:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.376268:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52c1c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2ad expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074761.376335:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075968:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074761.376421:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.378154:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.378330:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.379206:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616707328 00000100:00100000:1.0:1769074761.379212:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616707328 00000100:00100000:1.0:1769074761.379248:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4621 00000100:00100000:1.0:1769074761.379255:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62620:x1855007616707328:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074761.379511:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62620:x1855007616707328:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 256us (1180us total) trans 8589943852 rc 0/0 00000100:00100000:1.0:1769074761.379519:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4621 00000100:00100000:3.0:1769074761.383937:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616706304:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 16615us (17557us total) trans 21474841148 rc 0/0 00000100:00100000:3.0:1769074761.383952:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 696 00002000:00100000:3.0:1769074761.383965:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1593344->2097151) req@ffff89b5499d1180 x1855007616706304/t21474841148(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/448 e 0 to 0 dl 1769074772 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.383975:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1180 with x1855007616706304 ext(1593344->2097151) 00010000:00010000:3.0:1769074761.383984:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 21 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074761.384002:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 21 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:3.0:1769074761.384010:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5499d1180 00000100:00100000:0.0:1769074761.393621:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625075968:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074761.393661:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52c1c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ad expref: 20 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074761.393684:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866400/0x7fe2323adfd52c1c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ad expref: 20 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074761.393697:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:0.0:1769074761.393759:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616706688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 17524us (20967us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074761.393768:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13743 00000100:00100000:0.0:1769074761.396456:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.403101:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616707456 00000100:00100000:3.0:1769074761.403117:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616707456 00010000:00010000:3.0:1769074761.403171:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.403217:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6811 00000100:00100000:3.0:1769074761.403227:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616707456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074761.403242:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074761.403264:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074761.403279:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074761.403305:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52ba5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1593344->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b275 expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074761.403339:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.403342:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.403351:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.403361:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.403412:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074761.403428:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625076096:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074761.403540:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.404923:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.405119:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616707584 00000100:00100000:1.0:1769074761.405127:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616707584 00000100:00100000:1.0:1769074761.405176:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13744 00000100:00100000:1.0:1769074761.405185:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:62621:x1855007616707584:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.405200:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.405214:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2bb expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.405234:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074761.405236:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074761.405243:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.405262:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.405274:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866c00, rc 0) 00000100:00100000:1.0:1769074761.405330:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62621:x1855007616707584:12345-192.168.204.60@tcp:101:cat.0 Request processed in 146us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.405337:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13744 00000100:00100000:3.0:1769074761.416348:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625076096:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074761.416378:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074761.416456:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616707456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13229us (20020us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.416469:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6811 00000100:00100000:0.0:1769074761.424697:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.424862:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616707712 00000100:00100000:3.0:1769074761.424877:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616707712 00002000:00100000:3.0:1769074761.424944:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b549f2a680 x1855007616707712/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.424959:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2a680 with x1855007616707712 ext(1048576->2097151) 00002000:00010000:3.0:1769074761.424966:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f2a680 00000100:00100000:3.0:1769074761.425009:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8330 00000100:00100000:3.0:1769074761.425020:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616707712:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074761.427126:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.427440:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616707968 00000100:00100000:3.0:1769074761.427450:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616707968 00002000:00100000:3.0:1769074761.427512:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1589248->2097151) req@ffff89b5462dfb80 x1855007616707968/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.427523:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dfb80 with x1855007616707968 ext(1589248->2097151) 00002000:00010000:3.0:1769074761.427529:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462dfb80 00000100:00100000:3.0:1769074761.427562:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8331 00000100:00100000:3.0:1769074761.427571:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616707968:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074761.441699:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.441767:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.441853:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616707840 00000100:00100000:0.0:1769074761.441863:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616707840 00002000:00100000:0.0:1769074761.441915:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5462ded80 x1855007616707840/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.441927:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462ded80 with x1855007616707840 ext(1048576->2097151) 00002000:00010000:0.0:1769074761.441934:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462ded80 00000100:00100000:0.0:1769074761.442052:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708096 00000100:00100000:0.0:1769074761.442057:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708096 00002000:00100000:0.0:1769074761.442077:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2490368->2494463) req@ffff89b5462df800 x1855007616708096/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.442087:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462df800 with x1855007616708096 ext(2490368->2494463) 00002000:00010000:0.0:1769074761.442091:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5462df800 00000100:00100000:1.0:1769074761.442122:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8332 00000100:00100000:1.0:1769074761.442133:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616707840:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074761.443120:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616707968:12345-192.168.204.60@tcp:3:cat.0 Request processed in 15541us (15995us total) trans 0 rc 507904/507904 00000100:00100000:3.0:1769074761.443145:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8331 00002000:00100000:3.0:1769074761.443156:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1589248->2097151) req@ffff89b5462dfb80 x1855007616707968/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 507904/507904 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.443165:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dfb80 with x1855007616707968 ext(1589248->2097151) 00002000:00010000:3.0:1769074761.443171:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462dfb80 00000100:00100000:3.0:1769074761.443198:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8333 00000100:00100000:3.0:1769074761.443210:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616708096:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074761.443312:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2406:x1855007616708096:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 105us (1544us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074761.443319:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8333 00002000:00100000:3.0:1769074761.443325:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2490368->2494463) req@ffff89b5462df800 x1855007616708096/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/4536 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.443333:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462df800 with x1855007616708096 ext(2490368->2494463) 00002000:00010000:3.0:1769074761.443336:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5462df800 00000100:00100000:1.0:1769074761.451911:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616707712:12345-192.168.204.60@tcp:3:cat.0 Request processed in 26882us (27220us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074761.451925:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8330 00002000:00100000:1.0:1769074761.451937:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b549f2a680 x1855007616707712/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.451947:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2a680 with x1855007616707712 ext(1048576->2097151) 00002000:00010000:1.0:1769074761.451953:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f2a680 00000100:00100000:3.0:1769074761.459089:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616707840:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16945us (17386us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074761.459104:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8332 00002000:00100000:3.0:1769074761.459118:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5462ded80 x1855007616707840/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.459129:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462ded80 with x1855007616707840 ext(1048576->2097151) 00002000:00010000:3.0:1769074761.459135:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462ded80 00000100:00100000:3.0:1769074761.467399:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.467516:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708224 00000100:00100000:1.0:1769074761.467527:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708224 00000100:00100000:1.0:1769074761.467588:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13745 00000100:00100000:1.0:1769074761.467599:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62615:x1855007616708224:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.467620:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.467638:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2c2 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.467668:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.467675:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074761.467678:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074761.467682:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 10 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.467716:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 18 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.467730:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 18 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:1.0:1769074761.467743:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625076224:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074761.468003:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.473546:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708352 00000100:00100000:3.0:1769074761.473562:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708352 00000100:00100000:3.0:1769074761.473628:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708480 00000100:00100000:3.0:1769074761.473632:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708480 00000100:00100000:3.0:1769074761.473659:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13746 00000100:00100000:3.0:1769074761.473669:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616708352:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.473692:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.473715:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c31 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2c9 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074761.473779:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625076352:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074761.473918:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074761.474579:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13747 00000100:00100000:0.0:1769074761.474595:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616708480:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074761.474610:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074761.474625:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd52c38 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2d0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074761.474668:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625076480:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074761.474728:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.490723:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708608 00000100:00100000:3.0:1769074761.490733:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708608 00002000:00100000:3.0:1769074761.490781:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2834432->2838527) req@ffff89b5497a9880 x1855007616708608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.490792:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9880 with x1855007616708608 ext(2834432->2838527) 00002000:00010000:3.0:1769074761.490798:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a9880 00000100:00100000:3.0:1769074761.490860:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8334 00000100:00100000:3.0:1769074761.490869:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616708608:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074761.491015:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616708608:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 147us (427us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074761.491023:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8334 00002000:00100000:3.0:1769074761.491029:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2834432->2838527) req@ffff89b5497a9880 x1855007616708608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/4536 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.491038:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9880 with x1855007616708608 ext(2834432->2838527) 00002000:00010000:3.0:1769074761.491041:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a9880 00002000:00010000:0.0:1769074761.491533:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074760 -> 1769074761 00002000:00010000:0.0:1769074761.491539:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074760 -> 1769074761 00002000:00010000:0.0:1769074761.491542:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074760 -> 1769074761 00000100:00100000:0.0:1769074761.491558:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625076480:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074761.491600:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd52c38 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d0 expref: 20 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074761.491618:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd52c38 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d0 expref: 20 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074761.491629:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea5400, rc 0) 00000100:00100000:0.0:1769074761.491670:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616708480:12345-192.168.204.60@tcp:101:cat.0 Request processed in 17082us (18232us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074761.491677:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13747 00002000:00010000:3.0:1769074761.492057:0:27843:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074760 -> 1769074761 00002000:00010000:3.0:1769074761.492061:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074760 -> 1769074761 00002000:00010000:3.0:1769074761.492063:0:27843:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074760 -> 1769074761 00000100:00100000:3.0:1769074761.492074:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625076352:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074761.492092:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c31 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c9 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.492104:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c31 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c9 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.492113:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b6200, rc 0) 00000100:00100000:3.0:1769074761.492138:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616708352:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18471us (18770us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074761.492144:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13746 00000100:00100000:3.0:1769074761.500594:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.500677:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708736 00000100:00100000:0.0:1769074761.500690:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708736 00002000:00100000:0.0:1769074761.500739:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1589247) req@ffff89b544759880 x1855007616708736/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.500751:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759880 with x1855007616708736 ext(1048576->1589247) 00010000:00010000:0.0:1769074761.500760:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.500776:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 19 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:0.0:1769074761.500785:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b544759880 00000100:00100000:0.0:1769074761.500868:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 697 00000100:00100000:0.0:1769074761.500878:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616708736:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074761.506227:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.506528:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708864 00000100:00100000:0.0:1769074761.506539:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708864 00000100:00100000:0.0:1769074761.506596:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13748 00000100:00100000:0.0:1769074761.506607:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62621:x1855007616708864:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074761.506627:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074761.506646:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2d7 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.506677:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.506685:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074761.506687:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074761.506690:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.506722:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.506733:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 8 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.506746:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625076608:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074761.506857:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.507926:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625076224:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074761.507965:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 10 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.507987:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:1.0:1769074761.508052:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62615:x1855007616708224:12345-192.168.204.60@tcp:101:cat.0 Request processed in 40455us (40663us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.508061:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13745 00000100:00100000:1.0:1769074761.522030:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625076608:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074761.522075:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.522099:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4000, rc 0) 00000100:00100000:1.0:1769074761.522169:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62621:x1855007616708864:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15563us (15952us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.522179:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13748 00000100:00100000:0.0:1769074761.527416:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.528073:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616708992 00000100:00100000:1.0:1769074761.528086:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616708992 00002000:00100000:1.0:1769074761.528136:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2490368->3346431) req@ffff89b549f28000 x1855007616708992/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 504/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.528148:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28000 with x1855007616708992 ext(2490368->3346431) 00010000:00010000:1.0:1769074761.528157:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 9 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:1.0:1769074761.528171:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 9 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:1.0:1769074761.528180:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f28000 00000100:00100000:1.0:1769074761.528217:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 698 00000100:00100000:1.0:1769074761.528226:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616708992:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074761.539403:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616708992:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 11162us (11984us total) trans 17179874469 rc 0/0 00000100:00100000:1.0:1769074761.539427:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 698 00002000:00100000:1.0:1769074761.539447:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2490368->3346431) req@ffff89b549f28000 x1855007616708992/t17179874469(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 504/448 e 0 to 0 dl 1769074772 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.539460:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28000 with x1855007616708992 ext(2490368->3346431) 00010000:00010000:1.0:1769074761.539475:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 11 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:1.0:1769074761.539492:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 11 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:1.0:1769074761.539513:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f28000 00000100:00100000:0.0:1769074761.541162:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616708736:12345-192.168.204.60@tcp:4:cat.0 Request processed in 40274us (40566us total) trans 21474841149 rc 0/0 00000100:00100000:0.0:1769074761.541177:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 697 00002000:00100000:0.0:1769074761.541189:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1589247) req@ffff89b544759880 x1855007616708736/t21474841149(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/448 e 0 to 0 dl 1769074772 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.541199:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759880 with x1855007616708736 ext(1048576->1589247) 00010000:00010000:0.0:1769074761.541208:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 21 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.541220:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b640030a00/0x7fe2323adfd52bac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1589247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b27c expref: 21 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:0.0:1769074761.541228:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b544759880 00000100:00100000:0.0:1769074761.551793:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.554094:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616709248 00000100:00100000:3.0:1769074761.554110:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616709248 00000100:00100000:1.0:1769074761.554430:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.555591:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616709120 00000100:00100000:3.0:1769074761.555598:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616709120 00010000:00010000:3.0:1769074761.555620:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 11 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.562882:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6813 00000100:00100000:0.0:1769074761.562908:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616709120:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.562929:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.562954:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 11 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.562973:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 11 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.562999:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52b97 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3346431) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b267 expref: 11 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.563014:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074761.563016:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074761.563022:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.563033:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.563070:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074761.563083:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625076864:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074761.563162:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074761.570325:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.570371:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.570451:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616709376 00000100:00100000:0.0:1769074761.570463:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616709376 00002000:00100000:0.0:1769074761.570532:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5462dc700 x1855007616709376/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.570544:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dc700 with x1855007616709376 ext(1048576->2097151) 00002000:00010000:0.0:1769074761.570550:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462dc700 00000100:00100000:0.0:1769074761.570580:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616709504 00000100:00100000:0.0:1769074761.570583:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616709504 00002000:00100000:0.0:1769074761.570599:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5462dd180 x1855007616709504/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.570607:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dd180 with x1855007616709504 ext(1048576->2097151) 00002000:00010000:0.0:1769074761.570611:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5462dd180 00000100:00100000:0.0:1769074761.570636:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8335 00000100:00100000:0.0:1769074761.570646:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616709376:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074761.579523:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625076864:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074761.579547:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.579612:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616709120:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16717us (25185us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.579623:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6813 00000100:00100000:3.0:1769074761.580099:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616709632 00000100:00100000:3.0:1769074761.580109:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616709632 00002000:00100000:3.0:1769074761.580164:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5497a8000 x1855007616709632/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.580174:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8000 with x1855007616709632 ext(2097152->3145727) 00002000:00010000:3.0:1769074761.580181:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a8000 00000100:00100000:3.0:1769074761.580220:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8337 00000100:00100000:3.0:1769074761.580228:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616709632:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074761.592440:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616709504:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 21159us (22058us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074761.592454:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8336 00002000:00100000:3.0:1769074761.592467:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5462dd180 x1855007616709504/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.592478:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dd180 with x1855007616709504 ext(1048576->2097151) 00002000:00010000:3.0:1769074761.592484:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5462dd180 00000100:00100000:1.0:1769074761.670448:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616709376:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 99786us (100135us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074761.670466:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8335 00002000:00100000:1.0:1769074761.670483:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5462dc700 x1855007616709376/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.670503:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dc700 with x1855007616709376 ext(1048576->2097151) 00002000:00010000:1.0:1769074761.670509:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462dc700 00000100:00100000:0.0:1769074761.676486:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.676725:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616709888 00000100:00100000:1.0:1769074761.676740:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616709888 00000100:00100000:1.0:1769074761.676858:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13252 00000100:00100000:1.0:1769074761.676870:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62626:x1855007616709888:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074761.676903:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.676929:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52c46 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2de expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.677030:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677046:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677058:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677073:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677080:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677085:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677216:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52c5b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677229:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52c5b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677236:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52c5b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074761.677246:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074761.677340:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677351:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677356:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677362:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677367:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d96000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074761.677372:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677379:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677392:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074761.677394:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074761.677399:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677405:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52c4d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677419:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677425:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677430:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677434:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d96e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074761.677438:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677443:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677449:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294700) 00010000:00010000:1.0:1769074761.677452:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074761.677454:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677458:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52c54 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074761.677511:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52c46 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2de expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.677524:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52c5b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2de expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.677531:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074761.677533:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074761.677536:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97400, rc 0) 00000100:00100000:1.0:1769074761.677598:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62626:x1855007616709888:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 730us (1123us total) trans 8589943853 rc 0/0 00000100:00100000:1.0:1769074761.677606:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13252 00000100:00100000:1.0:1769074761.690007:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710144 00000100:00100000:1.0:1769074761.690020:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710144 00000100:00100000:1.0:1769074761.690096:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4622 00000100:00100000:1.0:1769074761.690107:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62626:x1855007616710144:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074761.690424:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62626:x1855007616710144:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 318us (1019us total) trans 8589943854 rc 0/0 00000100:00100000:1.0:1769074761.690432:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4622 00000100:00100000:0.0:1769074761.692125:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710272 00000100:00100000:0.0:1769074761.692141:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710272 00000100:00100000:0.0:1769074761.692221:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 986 00000100:00100000:0.0:1769074761.692230:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616710272:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.692245:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.692270:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52c5b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2de expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074761.692291:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074761.692297:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.692361:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616710272:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 132us (2830us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.692368:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 986 00000100:00100000:0.0:1769074761.702723:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616709632:12345-192.168.204.60@tcp:3:cat.0 Request processed in 122478us (123326us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074761.702742:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8337 00002000:00100000:0.0:1769074761.702758:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5497a8000 x1855007616709632/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.702771:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8000 with x1855007616709632 ext(2097152->3145727) 00002000:00010000:0.0:1769074761.702779:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a8000 00000100:00100000:1.0:1769074761.708433:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710400 00000100:00100000:1.0:1769074761.708443:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710400 00000100:00100000:1.0:1769074761.708515:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13749 00000100:00100000:1.0:1769074761.708525:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62615:x1855007616710400:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074761.708543:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.708562:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2e5 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.708587:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074761.708589:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074761.708597:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.708613:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.708627:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97800, rc 0) 00000100:00100000:1.0:1769074761.708692:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62615:x1855007616710400:12345-192.168.204.60@tcp:101:cat.0 Request processed in 168us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.708699:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13749 00000100:00100000:3.0:1769074761.731689:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710528 00000100:00100000:3.0:1769074761.731700:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710528 00000100:00100000:3.0:1769074761.731762:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13750 00000100:00100000:3.0:1769074761.731773:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62621:x1855007616710528:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.731792:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.731859:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2ec expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.731891:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.731899:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074761.731901:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074761.731905:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 21 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.731941:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.731954:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 9 pid: 27843 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.731967:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625076992:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.732051:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.744284:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625076992:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.744346:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 21 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.744373:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b6200, rc 0) 00000100:00100000:3.0:1769074761.744456:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62621:x1855007616710528:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12684us (12864us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.744467:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13750 00000100:00100000:0.0:1769074761.753856:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710656 00000100:00100000:0.0:1769074761.753866:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710656 00002000:00100000:0.0:1769074761.753914:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2953216->3145727) req@ffff89b5497abb80 x1855007616710656/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.753926:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497abb80 with x1855007616710656 ext(2953216->3145727) 00010000:00010000:0.0:1769074761.753934:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 10 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.753950:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 10 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:0.0:1769074761.753958:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5497abb80 00000100:00100000:0.0:1769074761.753997:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 699 00000100:00100000:0.0:1769074761.754007:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616710656:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074761.770924:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616710656:12345-192.168.204.60@tcp:4:cat.0 Request processed in 16832us (17207us total) trans 21474841150 rc 0/0 00000100:00100000:0.0:1769074761.770938:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 699 00002000:00100000:0.0:1769074761.770952:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2953216->3145727) req@ffff89b5497abb80 x1855007616710656/t21474841150(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/448 e 0 to 0 dl 1769074772 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.770963:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497abb80 with x1855007616710656 ext(2953216->3145727) 00010000:00010000:0.0:1769074761.770972:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.770990:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:0.0:1769074761.771004:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5497abb80 00000100:00100000:3.0:1769074761.772510:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074761.772649:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.773080:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710912 00000100:00100000:1.0:1769074761.773093:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710912 00002000:00100000:1.0:1769074761.773147:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54475b800 x1855007616710912/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 504/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.773166:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475b800 with x1855007616710912 ext(2097152->4194303) 00010000:00010000:1.0:1769074761.773175:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:1.0:1769074761.773190:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:1.0:1769074761.773198:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54475b800 00000100:00100000:1.0:1769074761.773227:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 700 00000100:00100000:1.0:1769074761.773237:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616710912:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074761.774441:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616710784 00000100:00100000:3.0:1769074761.774451:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616710784 00002000:00100000:3.0:1769074761.774485:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54475ad80 x1855007616710784/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074761.774512:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475ad80 with x1855007616710784 ext(2097152->4194303) 00002000:00010000:3.0:1769074761.774517:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54475ad80 00000100:00100000:3.0:1769074761.774550:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8338 00000100:00100000:3.0:1769074761.774558:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616710784:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074761.831187:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616710912:12345-192.168.204.60@tcp:3:cat.0 Request processed in 57940us (58688us total) trans 0 rc 1904640/1904640 00000100:00100000:1.0:1769074761.831202:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 700 00002000:00100000:1.0:1769074761.831215:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54475b800 x1855007616710912/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 504/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:H/600/0 rc 1904640/1904640 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.831226:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475b800 with x1855007616710912 ext(2097152->4194303) 00010000:00010000:1.0:1769074761.831236:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 1849 lvb_type: 0 00010000:00010000:1.0:1769074761.831257:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2953216->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b244 expref: 12 pid: 27843 timeout: 1849 lvb_type: 0 00002000:00010000:1.0:1769074761.831266:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54475b800 00000100:00100000:3.0:1769074761.867283:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616711168 00000100:00100000:3.0:1769074761.867298:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616711168 00000100:00100000:3.0:1769074761.867389:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13751 00000100:00100000:3.0:1769074761.867399:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62615:x1855007616711168:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.867422:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.867449:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2f3 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.867486:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074761.867503:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 13 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.867516:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 13 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.867529:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b7e00, rc 0) 00000100:00100000:3.0:1769074761.867609:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62615:x1855007616711168:12345-192.168.204.60@tcp:101:cat.0 Request processed in 211us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.867618:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13751 00000100:00100000:0.0:1769074761.873219:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.893130:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.920075:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.920123:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.920264:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616711424 00000100:00100000:0.0:1769074761.920278:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616711424 00002000:00100000:0.0:1769074761.920328:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5462dd500 x1855007616711424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.920340:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dd500 with x1855007616711424 ext(2097152->4194303) 00002000:00010000:0.0:1769074761.920346:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462dd500 00000100:00100000:0.0:1769074761.920377:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616711552 00000100:00100000:0.0:1769074761.920380:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616711552 00002000:00100000:0.0:1769074761.920395:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5462dca80 x1855007616711552/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/0 e 0 to 0 dl 1769074772 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.920403:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dca80 with x1855007616711552 ext(3145728->4194303) 00002000:00010000:0.0:1769074761.920406:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5462dca80 00000100:00100000:0.0:1769074761.920431:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8340 00000100:00100000:0.0:1769074761.920440:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616711424:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074761.962393:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616711424:12345-192.168.204.60@tcp:3:cat.0 Request processed in 41939us (42315us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074761.962409:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8340 00002000:00100000:0.0:1769074761.962424:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5462dd500 x1855007616711424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.962435:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dd500 with x1855007616711424 ext(2097152->4194303) 00002000:00010000:0.0:1769074761.962442:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5462dd500 00000100:00100000:0.0:1769074761.963364:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616711552:12345-192.168.204.60@tcp:3:cat.0 Request processed in 42679us (43235us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074761.963374:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8341 00002000:00100000:0.0:1769074761.963382:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5462dca80 x1855007616711552/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074761.963390:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dca80 with x1855007616711552 ext(3145728->4194303) 00002000:00010000:0.0:1769074761.963396:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5462dca80 00000100:00100000:0.0:1769074761.963900:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.972049:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616711808 00000100:00100000:1.0:1769074761.972061:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616711808 00000100:00100000:1.0:1769074761.972116:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13752 00000100:00100000:1.0:1769074761.972125:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62624:x1855007616711808:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074761.972144:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074761.972163:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b2fa expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.972193:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 22 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.972202:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 11 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.972209:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074761.972211:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074761.972215:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3334144->4161535] (req 3334144->4161535) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.972248:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 11 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.972260:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 11 pid: 28519 timeout: 1849 lvb_type: 0 00000100:00100000:1.0:1769074761.972274:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625077376:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074761.972371:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 22 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.972381:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 22 pid: 28519 timeout: 1849 lvb_type: 0 00000100:00100000:1.0:1769074761.972389:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625077504:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074761.972409:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f56c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074761.983933:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.983982:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.984274:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616711680:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19671us (20373us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074761.984288:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8342 00002000:00100000:1.0:1769074761.984302:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b549f2ad80 x1855007616711680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:297/0 lens 488/440 e 0 to 0 dl 1769074772 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074761.984322:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2ad80 with x1855007616711680 ext(4194304->4808703) 00002000:00010000:1.0:1769074761.984330:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549f2ad80 00000100:00100000:3.0:1769074761.984496:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616711936 00000100:00100000:3.0:1769074761.984505:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616711936 00000100:00100000:3.0:1769074761.984551:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712064 00000100:00100000:3.0:1769074761.984555:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712064 00000100:00100000:3.0:1769074761.984580:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13753 00000100:00100000:3.0:1769074761.984591:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62626:x1855007616711936:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074761.984610:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.984630:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b301 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.984658:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.984668:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.984675:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.984678:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.984682:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1056768->1802239] (req 1056768->1802239) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.984722:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.984734:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 23 pid: 28955 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.984747:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625077632:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074761.984943:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.984952:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 13 pid: 28955 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074761.984967:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625077760:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074761.984997:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074761.985017:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13754 00000100:00100000:3.0:1769074761.985024:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616712064:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074761.985032:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074761.985041:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b308 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074761.985055:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.985057:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074761.985064:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b308 expref: 24 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.985078:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b308 expref: 24 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074761.985088:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b7200, rc 0) 00000100:00100000:3.0:1769074761.985109:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616712064:12345-192.168.204.60@tcp:101:cat.0 Request processed in 86us (1128us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074761.985115:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13754 00000100:00100000:1.0:1769074761.994221:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625077376:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074761.994237:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625077504:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074761.994279:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3334144->4161535] (req 3334144->4161535) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074761.994297:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640031200, rc 0) 00000100:00100000:1.0:1769074761.994373:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62624:x1855007616711808:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 22248us (22494us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074761.994382:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13752 00000100:00100000:1.0:1769074761.999423:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.999455:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.999475:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074761.999508:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074761.999574:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712192 00000100:00100000:0.0:1769074761.999584:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712192 00010000:00010000:0.0:1769074761.999618:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 23 pid: 28519 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.999654:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712320 00000100:00100000:0.0:1769074761.999657:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712320 00010000:00010000:0.0:1769074761.999667:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 24 pid: 28955 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.999683:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712448 00000100:00100000:0.0:1769074761.999685:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712448 00010000:00010000:0.0:1769074761.999694:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 14 pid: 28519 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.999704:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712576 00000100:00100000:0.0:1769074761.999707:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712576 00010000:00010000:0.0:1769074761.999716:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 15 pid: 28955 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074761.999729:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6815 00000100:00100000:0.0:1769074761.999736:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616712192:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.999751:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.999771:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 24 pid: 28519 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.999783:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 24 pid: 28519 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.999828:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd52c70 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2ec expref: 24 pid: 28519 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.999855:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074761.999909:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616712192:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 172us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074761.999917:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6815 00000100:00100000:0.0:1769074761.999936:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6816 00000100:00100000:0.0:1769074761.999942:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616712320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074761.999952:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074761.999961:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 22 pid: 28955 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.999975:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 22 pid: 28955 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074761.999987:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.000005:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616712320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 64us (550us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.000010:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6816 00000100:00100000:0.0:1769074762.000022:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6817 00000100:00100000:0.0:1769074762.000028:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616712448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.000036:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.000043:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 15 pid: 28519 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074762.000052:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 15 pid: 28519 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074762.000064:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd52c77 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2f3 expref: 15 pid: 28519 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074762.000075:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.000077:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.000084:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.000094:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.000128:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.000142:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625077888:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074762.000187:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074762.000497:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6818 00000100:00100000:1.0:1769074762.000506:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616712576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074762.000515:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074762.000527:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 14 pid: 28955 timeout: 1849 lvb_type: 0 00010000:00010000:1.0:1769074762.000537:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd52be4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b298 expref: 14 pid: 28955 timeout: 1849 lvb_type: 0 00010000:00010000:1.0:1769074762.000550:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.000552:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.000556:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.000563:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.000583:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074762.000593:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625078016:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074762.000632:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f56c0 going to sleep for 11 seconds 00010000:00010000:3.0:1769074762.002157:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd52c23 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2bb expref: 21 pid: 28955 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074762.002178:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625077632:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.004260:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.004301:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.004358:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.004390:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712960 00000100:00100000:1.0:1769074762.004399:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712960 00000100:00100000:3.0:1769074762.004456:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712704 00000100:00100000:3.0:1769074762.004461:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712704 00000100:00100000:3.0:1769074762.004511:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616712832 00000100:00100000:3.0:1769074762.004514:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616712832 00000100:00100000:3.0:1769074762.004537:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13755 00000100:00100000:3.0:1769074762.004545:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616712704:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.004564:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.004581:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52c93 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b30f expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.004643:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625078144:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074762.004719:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.004739:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13756 00000100:00100000:3.0:1769074762.004747:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616712832:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.004754:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.004762:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52c9a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b316 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.004788:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625078272:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074762.004846:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074762.005212:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625077888:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074762.005231:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.005271:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616712448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5242us (5794us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.005281:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6817 00000100:00100000:3.0:1769074762.010368:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625078016:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074762.010399:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.010467:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616712576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9961us (10956us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.010479:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6818 00002000:00010000:3.0:1769074762.010605:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074760 -> 1769074761 00002000:00010000:3.0:1769074762.010610:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074760 -> 1769074761 00002000:00010000:3.0:1769074762.010613:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074760 -> 1769074761 00000100:00100000:3.0:1769074762.010631:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625078144:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074762.010659:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52c93 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b30f expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.010683:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd52c93 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b30f expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.010696:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:3.0:1769074762.010780:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616712704:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6235us (6529us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074762.010787:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13755 00000100:00100000:3.0:1769074762.010889:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625078272:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074762.010904:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52c9a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b316 expref: 14 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.010915:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52c9a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b316 expref: 14 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.010923:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:3.0:1769074762.010944:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616712832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6198us (6642us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074762.010950:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13756 00000100:00100000:3.0:1769074762.011057:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.011194:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616713088 00000100:00100000:0.0:1769074762.011202:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616713088 00002000:00100000:0.0:1769074762.011242:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5446c1f80 x1855007616713088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074762.011251:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1f80 with x1855007616713088 ext(4194304->4808703) 00002000:00010000:0.0:1769074762.011255:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c1f80 00000100:00100000:0.0:1769074762.011281:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8343 00000100:00100000:0.0:1769074762.011290:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616713088:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074762.023004:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.024680:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.024722:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616713344 00000100:00100000:3.0:1769074762.024728:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616713344 00002000:00100000:3.0:1769074762.029398:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1056768->1060863) req@ffff89b5446c0a80 x1855007616713344/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.029410:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0a80 with x1855007616713344 ext(1056768->1060863) 00002000:00010000:3.0:1769074762.029416:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c0a80 00000100:00100000:0.0:1769074762.034543:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625078400:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074762.034596:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52ca1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b31d expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.034618:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52ca1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b31d expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.034631:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cf6c800, rc 0) 00000100:00100000:3.0:1769074762.034643:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.034677:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616713472 00000100:00100000:0.0:1769074762.034681:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616713216:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11351us (11686us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074762.034684:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616713472 00000100:00100000:0.0:1769074762.034689:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13757 00000100:00100000:3.0:1769074762.034733:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13758 00000100:00100000:3.0:1769074762.034742:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616713472:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.034760:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.034775:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52ca8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b324 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00100000:1.0:1769074762.034825:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3334144->3338239) req@ffff89b54759f100 x1855007616712960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.034836:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f100 with x1855007616712960 ext(3334144->3338239) 00002000:00010000:1.0:1769074762.034840:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759f100 00000100:00100000:3.0:1769074762.034840:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625078528:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074762.034861:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8345 00000100:00100000:1.0:1769074762.034868:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616712960:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074762.034899:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074762.034977:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616712960:12345-192.168.204.60@tcp:3:cat.0 Request processed in 110us (30619us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074762.034985:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8345 00002000:00100000:1.0:1769074762.034990:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3334144->3338239) req@ffff89b54759f100 x1855007616712960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/4536 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.034998:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f100 with x1855007616712960 ext(3334144->3338239) 00002000:00010000:1.0:1769074762.035000:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759f100 00000100:00100000:0.0:1769074762.039934:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625078528:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074762.039974:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52ca8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b324 expref: 22 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.039998:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52ca8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b324 expref: 22 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.040010:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:0.0:1769074762.040086:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616713472:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5343us (5441us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074762.040096:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13758 00000100:00100000:1.0:1769074762.041369:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.057155:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.057676:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616713856 00000100:00100000:0.0:1769074762.057710:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616713856 00000100:00100000:0.0:1769074762.057862:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13759 00000100:00100000:0.0:1769074762.057879:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616713856:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074762.057925:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.057959:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52caf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b32b expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.058036:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625078656:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074762.058146:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.078433:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.078576:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.078666:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616713984 00000100:00100000:1.0:1769074762.078677:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616713984 00002000:00100000:1.0:1769074762.078728:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1798144->1802239) req@ffff89b5446c1180 x1855007616713984/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.078739:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1180 with x1855007616713984 ext(1798144->1802239) 00002000:00010000:1.0:1769074762.078746:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c1180 00000100:00100000:1.0:1769074762.078772:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616714112 00000100:00100000:1.0:1769074762.078776:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616714112 00002000:00100000:1.0:1769074762.078790:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (4157440->4161535) req@ffff89b5446c0700 x1855007616714112/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.078797:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0700 with x1855007616714112 ext(4157440->4161535) 00002000:00010000:1.0:1769074762.078800:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c0700 00000100:00100000:1.0:1769074762.078832:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8346 00000100:00100000:0.0:1769074762.078833:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625078656:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074762.078840:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616713984:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:0.0:1769074762.078866:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52caf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b32b expref: 23 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.078885:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52caf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b32b expref: 23 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.078896:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8400, rc 0) 00000100:00100000:0.0:1769074762.078945:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616713856:12345-192.168.204.60@tcp:101:cat.0 Request processed in 21071us (21805us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074762.078952:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13759 00000100:00100000:1.0:1769074762.078953:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616713984:12345-192.168.204.60@tcp:3:cat.0 Request processed in 114us (531us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074762.078959:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8346 00002000:00100000:1.0:1769074762.078965:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1798144->1802239) req@ffff89b5446c1180 x1855007616713984/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/4536 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.078974:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1180 with x1855007616713984 ext(1798144->1802239) 00002000:00010000:1.0:1769074762.078977:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c1180 00000100:00100000:1.0:1769074762.078990:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8347 00000100:00100000:1.0:1769074762.078997:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616714112:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074762.079053:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616714112:12345-192.168.204.60@tcp:3:cat.0 Request processed in 56us (477us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074762.079059:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8347 00002000:00100000:1.0:1769074762.079065:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (4157440->4161535) req@ffff89b5446c0700 x1855007616714112/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/4536 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.079072:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0700 with x1855007616714112 ext(4157440->4161535) 00002000:00010000:1.0:1769074762.079075:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c0700 00010000:00010000:0.0:1769074762.141222:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074762.143036:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.143970:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616714240 00000100:00100000:0.0:1769074762.143986:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616714240 00000100:00100000:0.0:1769074762.144085:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13253 00000100:00100000:0.0:1769074762.144097:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62631:x1855007616714240:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074762.144126:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.144149:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd52cb6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b332 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.144225:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd52cbd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144239:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd52cbd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144249:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd52cbd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144265:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52cc4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144271:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52cc4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144276:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd52cc4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144365:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144384:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.144395:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.144406:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 5/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.144461:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.144479:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 223 pid: 6512 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074762.144506:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625078784:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.144678:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.144699:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 14 pid: 6512 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074762.144714:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625078912:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.144759:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074762.150050:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.150933:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625078784:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.150944:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625078912:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.150969:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00080000:3.0:1769074762.151086:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074762.151773:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.154690:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.154757:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616714880 00000100:00100000:3.0:1769074762.154770:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616714880 00010000:00010000:3.0:1769074762.154875:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 225 pid: 6512 timeout: 1849 lvb_type: 0 00000100:00100000:3.0:1769074762.154909:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6819 00000100:00100000:3.0:1769074762.154918:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:2409:x1855007616714880:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.154937:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.154943:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 225 pid: 6512 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074762.154957:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 225 pid: 6512 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074762.154981:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd529ad lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b110 expref: 225 pid: 6512 timeout: 1849 lvb_type: 0 00010000:00010000:3.0:1769074762.155000:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.155006:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.155020:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.155026:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.155032:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.155115:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2409:x1855007616714880:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 198us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.155124:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6819 00000100:00100000:3.0:1769074762.157966:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.158072:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616715008 00000100:00100000:0.0:1769074762.158083:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616715008 00010000:00010000:0.0:1769074762.158126:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 16 pid: 6512 timeout: 1849 lvb_type: 0 00000100:00100000:0.0:1769074762.158170:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6820 00000100:00100000:0.0:1769074762.158180:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effe050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616715008:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.158211:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.158218:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 16 pid: 6512 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074762.158231:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 16 pid: 6512 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074762.158257:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd52959 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b0ca expref: 16 pid: 6512 timeout: 1849 lvb_type: 0 00010000:00010000:0.0:1769074762.158283:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.158288:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.158300:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.158320:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.158363:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.158429:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effe050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616715008:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 249us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.158436:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6820 00000100:00100000:3.0:1769074762.164907:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.165091:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616715136 00000100:00100000:0.0:1769074762.165102:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616715136 00000100:00100000:0.0:1769074762.165176:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13760 00000100:00100000:0.0:1769074762.165185:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2409:x1855007616715136:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074762.165216:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.165240:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52cd2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b339 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.165613:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625079040:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074762.165713:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984e40 going to sleep for 11 seconds 00000100:00080000:3.0:1769074762.165901:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074762.167093:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.170550:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.170640:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616715392 00000100:00100000:3.0:1769074762.170649:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616715392 00000100:00100000:0.0:1769074762.170695:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.170708:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616715520 00000100:00100000:3.0:1769074762.170712:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616715520 00000100:00100000:0.0:1769074762.170730:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.170903:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625079040:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074762.170937:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52cd2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b339 expref: 10 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.170959:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd52cd2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b339 expref: 10 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074762.170972:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9400, rc 0) 00000100:00100000:0.0:1769074762.171029:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616715136:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 5845us (6134us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074762.171036:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13760 00000100:00100000:3.0:1769074762.171170:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616715648 00000100:00100000:3.0:1769074762.171175:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616715648 00000100:00100000:3.0:1769074762.171213:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13762 00000100:00100000:3.0:1769074762.171222:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616715392:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074762.171237:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.171253:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b347 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.171271:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.171273:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.171281:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.171298:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.171463:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89600, rc 0) 00000100:00100000:3.0:1769074762.171501:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616715392:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 280us (960us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.171507:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13762 00000100:00100000:3.0:1769074762.171522:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13763 00000100:00100000:3.0:1769074762.171529:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616715520:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074762.171536:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.171545:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b34e expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.171556:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.171558:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.171563:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.171573:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.171582:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:3.0:1769074762.171602:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616715520:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 75us (910us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.171608:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13763 00000100:00100000:3.0:1769074762.171621:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13764 00000100:00100000:3.0:1769074762.171627:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616715648:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074762.171634:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.171643:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52cee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b355 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.171691:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625079168:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074762.171738:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00080000:1.0:1769074762.171791:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074762.174271:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625079168:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074762.174308:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52cee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b355 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.174337:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52cee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b355 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.174349:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:3.0:1769074762.174405:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616715648:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 2777us (3675us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074762.174414:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13764 00000100:00100000:1.0:1769074762.177939:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.193982:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.194063:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616716032 00000100:00100000:3.0:1769074762.194076:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616716032 00010000:00010000:3.0:1769074762.194122:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b332 expref: 11 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.194230:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6821 00000100:00100000:3.0:1769074762.194241:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616716032:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.194259:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.194266:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b332 expref: 11 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.194281:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd52ccb lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b332 expref: 11 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.194318:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.194324:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cf6d600/0x7fe2323adfd52cfc lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.194335:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cf6d600/0x7fe2323adfd52cfc lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.194344:0:6499:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cf6d600/0x7fe2323adfd52cfc lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.194364:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.194429:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616716032:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 189us (457us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.194450:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6821 00000100:00100000:0.0:1769074762.200405:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.217447:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.217499:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.217522:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.217559:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616716416 00000100:00100000:1.0:1769074762.217579:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616716416 00000100:00100000:1.0:1769074762.217635:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616716544 00000100:00100000:1.0:1769074762.217639:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616716544 00000100:00100000:1.0:1769074762.217656:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616716672 00000100:00100000:1.0:1769074762.217658:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616716672 00000100:00100000:1.0:1769074762.217679:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13765 00000100:00100000:1.0:1769074762.217687:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616716416:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:1.0:1769074762.217730:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.217750:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 20 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.217761:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.217770:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 10 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.217776:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.217779:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.217782:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 6/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.217850:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 10 pid: 9048 timeout: 0 lvb_type: 1 00000100:00100000:3.0:1769074762.218025:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13766 00000100:00100000:3.0:1769074762.218035:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616716544:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:3.0:1769074762.218058:0:27844:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218071:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218077:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074762.218078:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074762.218081:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218102:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218111:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 10 pid: 28955 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.218125:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625079552:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.218210:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.218232:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13767 00000100:00100000:3.0:1769074762.218240:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616716672:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:3.0:1769074762.218259:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218269:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 10 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218277:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 10 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218284:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b308 expref: 20 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218290:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.218291:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.218293:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 6/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218309:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b308 expref: 20 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218318:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b308 expref: 20 pid: 28954 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.218328:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625079680:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.218361:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 10 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218371:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 10 pid: 28954 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.218379:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625079808:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.218404:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 10 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218413:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 10 pid: 28519 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.218421:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625079936:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.218438:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.218514:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.218534:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616716800 00000100:00100000:3.0:1769074762.218539:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616716800 00000100:00100000:3.0:1769074762.218579:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13768 00000100:00100000:3.0:1769074762.218586:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b35b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616716800:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:3.0:1769074762.218607:0:28519:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218618:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218629:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218636:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218642:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.218643:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.218646:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 6/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218664:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218672:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.218682:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:28519:1855007625080064:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.218717:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.218725:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 13 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.218734:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:28519:1855007625080192:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.218766:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.218775:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.218782:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:28519:1855007625080320:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.218801:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.219017:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.219034:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616716928 00000100:00100000:3.0:1769074762.219038:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616716928 00000100:00100000:3.0:1769074762.219065:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13769 00000100:00100000:3.0:1769074762.219072:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616716928:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:3.0:1769074762.219091:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219103:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219110:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.219116:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219122:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.219124:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.219126:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 6/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219142:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219150:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 10 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.219159:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625080448:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.219190:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.219198:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 14 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.219206:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625080576:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.219233:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219243:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.219251:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625080704:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.219266:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653746840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.219313:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.219327:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616717056 00000100:00100000:3.0:1769074762.219331:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616717056 00000100:00100000:3.0:1769074762.219355:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13770 00000100:00100000:3.0:1769074762.219361:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616717056:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:3.0:1769074762.219379:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219389:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219394:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074762.219395:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074762.219398:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219415:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.219425:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.219434:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625080832:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.219452:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397900 going to sleep for 11 seconds 00010000:00010000:1.0:1769074762.219984:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 10 pid: 9048 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.220001:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625079424:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.220046:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.220055:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 14 pid: 9048 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.220063:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625080960:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.220091:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 20 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.220099:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 20 pid: 36516 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.220107:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625081088:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.220127:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.240476:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.240585:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:28519:1855007625080320:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.240639:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625080448:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.240681:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625080576:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.240713:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625080704:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.240746:0:9048:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.240799:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625080832:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.240858:0:8398:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.240929:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.240973:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625079424:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.242649:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625079552:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.242677:0:27844:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.244524:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625080960:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074762.244572:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625081088:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.244600:0:36516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.247573:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd52c8c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b308 expref: 20 pid: 28954 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.247598:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625079680:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.247613:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625079808:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.247623:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625079936:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.247649:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.272263:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616717312 00000100:00100000:0.0:1769074762.272275:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616717312 00010000:00010000:0.0:1769074762.272317:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 12 pid: 28954 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.272365:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6823 00000100:00100000:0.0:1769074762.272374:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616717312:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.272390:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.272413:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 12 pid: 28954 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.272427:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 12 pid: 28954 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.272452:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c200/0x7fe2323adfd52cd9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b340 expref: 12 pid: 28954 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.272471:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.272543:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616717312:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 170us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.272550:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6823 00000100:00100000:0.0:1769074762.272681:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.272703:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616717696 00000100:00100000:0.0:1769074762.272707:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616717696 00010000:00010000:0.0:1769074762.272722:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 21 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.272743:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6824 00000100:00100000:0.0:1769074762.272750:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616717696:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.272758:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.272772:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 21 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.272782:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 21 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.272802:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd52b35 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b221 expref: 21 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.272851:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.272875:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616717696:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 124us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.272882:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6824 00000100:00100000:3.0:1769074762.284233:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.284265:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616717824 00000100:00100000:3.0:1769074762.284274:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616717824 00010000:00010000:3.0:1769074762.284308:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 11 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.284344:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6825 00000100:00100000:3.0:1769074762.284353:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616717824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.284367:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.284385:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 11 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.284396:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 11 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.284414:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd52c69 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2e5 expref: 11 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.284429:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.284471:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616717824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 119us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.284478:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6825 00000100:00100000:3.0:1769074762.284599:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.284624:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.284641:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616717568 00000100:00100000:3.0:1769074762.284647:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616717568 00010000:00010000:3.0:1769074762.284663:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.284682:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6826 00000100:00100000:3.0:1769074762.284689:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2407:x1855007616717568:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.284698:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.284707:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:3.0:1769074762.284721:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:3.0:1769074762.284733:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd52ce7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b34e expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:3.0:1769074762.284743:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.284764:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616717568:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 75us (139us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.284770:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6826 00000100:00100000:3.0:1769074762.284793:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616717952 00000100:00100000:3.0:1769074762.284797:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616717952 00002000:00100000:3.0:1769074762.284857:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (3334144->4161535) req@ffff89b545388700 x1855007616717952/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.284866:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388700 with x1855007616717952 ext(3334144->4161535) 00010000:00010000:3.0:1769074762.284873:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.284884:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00002000:00010000:3.0:1769074762.284891:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545388700 00000100:00100000:3.0:1769074762.284914:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 702 00000100:00100000:3.0:1769074762.284922:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616717952:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074762.285085:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.285101:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718080 00000100:00100000:3.0:1769074762.285104:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718080 00010000:00010000:3.0:1769074762.285119:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.285136:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6827 00000100:00100000:3.0:1769074762.285142:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616718080:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.285149:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.285158:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.285167:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.285181:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4000/0x7fe2323adfd52c3f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2d7 expref: 20 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.285191:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.285192:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.285197:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285207:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285231:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.285251:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2407:x1855007616718080:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 109us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.285257:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6827 00010000:00010000:3.0:1769074762.285303:0:9048:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285311:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285329:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285337:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285345:0:9048:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285352:0:9048:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b546117000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074762.285356:0:9048:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285365:0:9048:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285376:0:9048:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.285388:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd52d3b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.285430:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616716928:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 66358us (66414us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.285439:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13769 00000100:00100000:3.0:1769074762.285516:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.285537:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718464 00000100:00100000:3.0:1769074762.285541:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718464 00010000:00010000:3.0:1769074762.285557:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 1850 lvb_type: 1 00000100:00100000:3.0:1769074762.285576:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6828 00000100:00100000:3.0:1769074762.285583:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616718464:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.285589:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.285599:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 1850 lvb_type: 1 00010000:00010000:3.0:1769074762.285607:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 1850 lvb_type: 1 00010000:00010000:3.0:1769074762.285619:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd52b6d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b23d expref: 10 pid: 27843 timeout: 1850 lvb_type: 1 00010000:00010000:3.0:1769074762.285628:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.285645:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616718464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 63us (130us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.285651:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6828 00000100:00100000:3.0:1769074762.290984:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616717440:12345-192.168.204.60@tcp:4:cat.0 Request processed in 47753us (50038us total) trans 21474841151 rc 0/0 00000100:00100000:3.0:1769074762.291001:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 701 00002000:00100000:3.0:1769074762.291014:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1056768->1802239) req@ffff89b54538b800 x1855007616717440/t21474841151(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/448 e 0 to 0 dl 1769074773 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.291025:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54538b800 with x1855007616717440 ext(1056768->1802239) 00010000:00010000:3.0:1769074762.291035:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 13 pid: 28955 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.291048:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 13 pid: 28955 timeout: 1850 lvb_type: 0 00002000:00010000:3.0:1769074762.291055:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54538b800 00000100:00100000:0.0:1769074762.292225:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.292256:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.292286:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.292318:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718208 00000100:00100000:0.0:1769074762.292327:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718208 00010000:00010000:0.0:1769074762.292362:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 13 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.292394:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718592 00000100:00100000:0.0:1769074762.292398:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718592 00010000:00010000:0.0:1769074762.292412:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 19 pid: 36516 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.292429:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718336 00000100:00100000:0.0:1769074762.292432:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718336 00010000:00010000:0.0:1769074762.292442:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 14 pid: 9048 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.292459:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6829 00000100:00100000:0.0:1769074762.292467:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616718208:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.292491:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.292512:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 14 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.292523:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 14 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.292545:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd52ce0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b347 expref: 14 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.292559:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.292561:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.292565:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292574:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292600:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.292649:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616718208:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 182us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.292657:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6829 00000100:00100000:0.0:1769074762.292674:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6830 00000100:00100000:0.0:1769074762.292681:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616718592:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.292689:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.292699:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 19 pid: 36516 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.292712:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 19 pid: 36516 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.292727:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd52642 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aeee expref: 19 pid: 36516 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.292739:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.292758:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616718592:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 78us (501us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.292764:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6830 00000100:00100000:0.0:1769074762.292776:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6831 00000100:00100000:0.0:1769074762.292782:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616718336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.292790:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.292798:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 12 pid: 9048 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.292838:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 12 pid: 9048 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.292850:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd52729 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af7a expref: 12 pid: 9048 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.292860:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.292878:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616718336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 96us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.292888:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6831 00010000:00010000:0.0:1769074762.292921:0:28519:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292930:0:28519:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292949:0:28519:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292957:0:28519:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292965:0:28519:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292972:0:28519:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b640030a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074762.292976:0:28519:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292985:0:28519:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.292995:0:28519:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.293002:0:28519:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd52d34 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.293048:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b35b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616716800:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 74461us (74534us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.293056:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13768 00000100:00100000:0.0:1769074762.293101:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.293121:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718720 00000100:00100000:0.0:1769074762.293125:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718720 00010000:00010000:0.0:1769074762.293141:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 9 pid: 9048 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.293160:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6832 00000100:00100000:0.0:1769074762.293166:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616718720:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.293174:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.293183:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 9 pid: 9048 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.293193:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 9 pid: 9048 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.293206:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3e00/0x7fe2323adfd52650 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879aefc expref: 9 pid: 9048 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.293216:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.293217:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.293221:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.293233:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.293250:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.293269:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2409:x1855007616718720:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 103us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.293275:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6832 00010000:00010000:3.0:1769074762.295380:0:36516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295398:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295424:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295431:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295443:0:36516:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295450:0:36516:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074762.295454:0:36516:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295462:0:36516:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295479:0:36516:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.295498:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd52d1f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.295587:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616716416:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 77897us (78147us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.295596:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13765 00000100:00100000:3.0:1769074762.297564:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718848 00000100:00100000:3.0:1769074762.297578:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718848 00010000:00010000:3.0:1769074762.297628:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 13 pid: 28519 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.297736:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6833 00000100:00100000:3.0:1769074762.297746:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616718848:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.297763:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.297782:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 13 pid: 28519 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.297796:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 13 pid: 28519 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.297837:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd52c2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2c2 expref: 13 pid: 28519 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.297855:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.297857:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.297862:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.297871:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.297910:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.297971:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616718848:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 227us (2245us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.297979:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6833 00010000:00010000:0.0:1769074762.298055:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298065:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298085:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298094:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298102:0:27843:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298109:0:27843:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074762.298113:0:27843:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298127:0:27843:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298139:0:27843:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.298145:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd52d2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.298194:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616716672:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 79955us (80671us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.298202:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13767 00000100:00100000:0.0:1769074762.318073:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.319894:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616718976 00000100:00100000:0.0:1769074762.319911:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616718976 00010000:00010000:0.0:1769074762.319953:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 12 pid: 28955 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.319998:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6834 00000100:00100000:0.0:1769074762.320007:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616718976:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.320022:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.320046:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 12 pid: 28955 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.320059:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 12 pid: 28955 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.320086:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7c00/0x7fe2323adfd52c85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1056768->1802239) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b301 expref: 12 pid: 28955 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.320100:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.320102:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.320107:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.320117:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.320156:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.320232:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616718976:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 224us (2167us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.320241:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6834 00010000:00010000:3.0:1769074762.320659:0:27844:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320672:0:27844:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320702:0:27844:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320711:0:27844:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320719:0:27844:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320727:0:27844:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074762.320731:0:27844:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320740:0:27844:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320755:0:27844:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.320763:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd52d26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.320888:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616716544:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 102844us (103377us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.320896:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13766 00000100:00100000:3.0:1769074762.321511:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616717952:12345-192.168.204.60@tcp:4:cat.0 Request processed in 36586us (36909us total) trans 21474841152 rc 0/0 00000100:00100000:3.0:1769074762.321522:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 702 00002000:00100000:3.0:1769074762.321533:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3334144->4161535) req@ffff89b545388700 x1855007616717952/t21474841152(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/448 e 0 to 0 dl 1769074773 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.321541:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388700 with x1855007616717952 ext(3334144->4161535) 00010000:00010000:3.0:1769074762.321548:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 22 pid: 27843 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.321574:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b640031200/0x7fe2323adfd52c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3334144->4161535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b2fa expref: 22 pid: 27843 timeout: 1850 lvb_type: 0 00002000:00010000:3.0:1769074762.321581:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545388700 00000100:00100000:1.0:1769074762.328614:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:3.0:1769074762.329116:0:8398:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329125:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329144:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329153:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329169:0:8398:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329181:0:8398:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074762.329185:0:8398:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329194:0:8398:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329205:0:8398:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.329211:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.329260:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2407:x1855007616717056:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 109898us (109945us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.329268:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13770 00000100:00100000:0.0:1769074762.336394:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616719360 00000100:00100000:0.0:1769074762.336403:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616719360 00000100:00100000:0.0:1769074762.336452:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13256 00000100:00100000:0.0:1769074762.336459:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62631:x1855007616719360:12345-192.168.204.60@tcp:36:mirror_io.0 00010000:00010000:0.0:1769074762.336529:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.336562:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd528db lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b076 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336581:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.336588:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074762.336635:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.336658:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336671:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336683:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336721:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 14 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336734:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336745:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.336758:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 9/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.337182:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.337211:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.337236:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081216:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.337478:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.337498:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 223 pid: 10283 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.337508:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081344:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.337540:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 14 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.337549:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 14 pid: 8855 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.337557:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081472:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.337585:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.337593:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.337600:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081600:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.337625:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.337632:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 14 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.337647:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081728:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.337673:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.337682:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.337701:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081856:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.337727:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074762.345366:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.345473:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.346223:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616719744 00000100:00100000:3.0:1769074762.346236:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616719744 00010000:00010000:3.0:1769074762.346280:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 224 pid: 10283 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.346477:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6836 00000100:00100000:3.0:1769074762.346497:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2409:x1855007616719744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.346516:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.346523:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 224 pid: 10283 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.346536:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 224 pid: 10283 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.346567:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.346572:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.346589:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.346664:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2409:x1855007616719744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 171us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.346672:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6836 00000100:00100000:0.0:1769074762.346793:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081216:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.346851:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030400/0x7fe2323adfd52419 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879adb3 expref: 223 pid: 10283 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.346869:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081344:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.346882:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081472:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.346891:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081600:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.348090:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.348111:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.348143:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616719872 00000100:00100000:1.0:1769074762.348151:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616719872 00010000:00010000:1.0:1769074762.348189:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 15 pid: 8855 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.348387:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720128 00000100:00100000:1.0:1769074762.348391:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720128 00010000:00010000:1.0:1769074762.348407:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 16 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.348427:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6837 00000100:00100000:1.0:1769074762.348436:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616719872:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074762.348522:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074762.348530:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 16 pid: 8855 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.348542:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 16 pid: 8855 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.348563:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd52404 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad9e expref: 16 pid: 8855 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.348578:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.348582:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.348592:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074762.348641:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616719872:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 206us (560us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.348648:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6837 00000100:00100000:1.0:1769074762.348667:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6838 00000100:00100000:1.0:1769074762.348673:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616720128:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074762.348680:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074762.348685:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 14 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.348695:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 14 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.348707:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.348711:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.348717:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074762.348736:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616720128:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 64us (625us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.348742:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6838 00000100:00100000:1.0:1769074762.348783:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.348803:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720000 00000100:00100000:1.0:1769074762.349562:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720000 00010000:00010000:1.0:1769074762.349594:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.349626:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6839 00000100:00100000:1.0:1769074762.349633:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616720000:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074762.349641:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074762.349646:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.349655:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.349676:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd523da lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad89 expref: 14 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.349691:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.349694:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.349701:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074762.349734:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616720000:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 102us (951us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.349740:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6839 00010000:00010000:0.0:1769074762.351603:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd526e3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af50 expref: 12 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.351626:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081728:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.351960:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625081856:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.351998:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074762.352017:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.352038:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720256 00000100:00100000:1.0:1769074762.352044:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720256 00010000:00010000:1.0:1769074762.352071:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.352091:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6840 00000100:00100000:1.0:1769074762.352097:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2408:x1855007616720256:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074762.352107:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074762.352112:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.352120:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.352135:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052000/0x7fe2323adfd526ab lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879af34 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.352143:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.352145:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.352151:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074762.353601:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2408:x1855007616720256:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1498us (1581us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.353612:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6840 00000100:00100000:0.0:1769074762.359025:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.359074:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720384 00000100:00100000:1.0:1769074762.359085:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720384 00010000:00010000:1.0:1769074762.359129:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.359220:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6841 00000100:00100000:1.0:1769074762.359230:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2408:x1855007616720384:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074762.359250:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074762.359259:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.359275:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.359310:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd52810 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b006 expref: 223 pid: 6513 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.359327:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.359334:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.359345:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.359352:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.359384:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074762.359479:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:2408:x1855007616720384:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 250us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.359506:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6841 00010000:00010000:0.0:1769074762.359597:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.359606:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000040:00080000:0.0:1769074762.359911:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2392, 88 off204704 00000004:00080000:0.0:1769074762.359922:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202114661ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.359964:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2394, 88 off204616 00000004:00080000:0.0:1769074762.359971:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202163968ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.359991:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2395, 88 off204704 00000004:00080000:0.0:1769074762.359996:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202188825ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.360059:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2394, 88 off204888 00000004:00080000:0.0:1769074762.360064:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202257028ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360073:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2395, 88 off204976 00000004:00080000:0.0:1769074762.360076:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202269373ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360095:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2397, 88 off204888 00000004:00080000:0.0:1769074762.360098:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202291153ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.360105:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2398, 88 off204976 00000004:00080000:0.0:1769074762.360108:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202300873ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.360537:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2399, 88 off205064 00000004:00080000:0.0:1769074762.360545:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202738165ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.360579:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2396, 88 off205064 00000004:00080000:0.0:1769074762.360583:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202776671ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360594:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2400, 88 off205152 00000004:00080000:0.0:1769074762.360597:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202790008ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.360606:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2397, 88 off205152 00000004:00080000:0.0:1769074762.360609:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202801931ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360617:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2401, 88 off205240 00000004:00080000:0.0:1769074762.360621:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202814253ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074762.360630:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2398, 88 off205240 00000004:00080000:0.0:1769074762.360634:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202826984ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360643:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2399, 88 off205328 00000004:00080000:0.0:1769074762.360646:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202838833ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360654:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2400, 88 off205416 00000004:00080000:0.0:1769074762.360657:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202850149ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360665:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2401, 88 off205504 00000004:00080000:0.0:1769074762.360668:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750202861295ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074762.360677:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2402, 88 off205328 00000004:00080000:0.0:1769074762.360680:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750202872694ns, next at 1706953247299ns, rc = 0 00000100:00100000:1.0:1769074762.363378:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625082368:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.363423:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.363494:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625082880:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.363521:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.363540:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625081984:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.363563:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625082240:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.363588:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1769074762.363603:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074762.363618:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625082496:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.363625:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.363648:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.363654:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625082752:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.363685:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625082240 00000100:00100000:1.0:1769074762.363685:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.363691:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625082240 00000100:00100000:1.0:1769074762.363711:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074762.363718:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625082496 00000100:00100000:1.0:1769074762.363722:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625082624:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.363723:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625082496 00000100:00100000:0.0:1769074762.363741:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625082752 00000100:00100000:0.0:1769074762.363746:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625082752 00000100:00100000:1.0:1769074762.363751:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.363759:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625082624 00000100:00100000:0.0:1769074762.363762:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625082624 00000100:00100000:0.0:1769074762.363778:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13772 00000100:00100000:0.0:1769074762.363785:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625082880:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.363787:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13773 00000100:00100000:1.0:1769074762.363801:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625081984:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074762.363963:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13774 00000100:00100000:3.0:1769074762.363972:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625082240:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074762.363989:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13775 00000100:00100000:3.0:1769074762.363994:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625082496:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074762.364135:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074762.364138:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.364140:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.364142:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.364144:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074762.364177:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625082880:12345-0@lo:2:osp-syn-0-0.0 Request processed in 393us (656us total) trans 21474841154 rc 0/0 00000100:00100000:3.0:1769074762.364347:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625082240:12345-0@lo:2:osp-syn-0-0.0 Request processed in 374us (724us total) trans 21474841155 rc 0/0 00000100:00100000:1.0:1769074762.364508:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625081984:12345-0@lo:2:osp-syn-0-0.0 Request processed in 711us (919us total) trans 21474841156 rc 0/0 00000100:00100000:0.0:1769074762.367951:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13772 00000100:00100000:0.0:1769074762.368014:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13777 00000100:00100000:0.0:1769074762.368036:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625082624:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074762.368193:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074762.368197:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.368200:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.368202:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.368205:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074762.368276:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625082624:12345-0@lo:2:osp-syn-0-0.0 Request processed in 253us (4525us total) trans 21474841158 rc 0/0 00000100:00100000:0.0:1769074762.368282:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13777 00000100:00100000:3.0:1769074762.368288:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13774 00000100:00100000:0.0:1769074762.368317:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625083008:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.368356:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:3.0:1769074762.368362:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:3.0:1769074762.368364:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:3.0:1769074762.368366:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769074762.368368:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:3.0:1769074762.368370:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074762.368387:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625083520:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074762.368397:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625082496:12345-0@lo:2:osp-syn-0-0.0 Request processed in 4402us (4748us total) trans 21474841159 rc 0/0 00000100:00100000:3.0:1769074762.368408:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13775 00000100:00100000:3.0:1769074762.368420:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625083008 00000100:00100000:0.0:1769074762.368422:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074762.368426:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625083008 00000100:00100000:0.0:1769074762.368453:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841134 gen 2 00000100:00100000:3.0:1769074762.368457:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13778 00000004:00080000:0.0:1769074762.368458:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5446c1500, transno 21474841111 00000100:00100000:3.0:1769074762.368463:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625083008:12345-0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.368466:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf96680, transno 21474841112 00000004:00080000:0.0:1769074762.368470:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf96a00, transno 21474841113 00000004:00080000:0.0:1769074762.368472:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf96300, transno 21474841114 00000004:00080000:0.0:1769074762.368475:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5446c2d80, transno 21474841115 00000004:00080000:0.0:1769074762.368477:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf96d80, transno 21474841116 00000004:00080000:0.0:1769074762.368479:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf95180, transno 21474841117 00000004:00080000:0.0:1769074762.368493:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf94700, transno 21474841118 00000004:00080000:0.0:1769074762.368495:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54ce9e680, transno 21474841119 00000004:00080000:0.0:1769074762.368499:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54538ad80, transno 21474841120 00000004:00080000:0.0:1769074762.368501:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cfefb80, transno 21474841121 00000004:00080000:0.0:1769074762.368504:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54ce9f480, transno 21474841122 00000004:00080000:0.0:1769074762.368506:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cfef100, transno 21474841123 00000004:00080000:0.0:1769074762.368508:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54ce9d180, transno 21474841124 00000004:00080000:0.0:1769074762.368510:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cfee300, transno 21474841125 00000004:00080000:0.0:1769074762.368513:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54ce9d500, transno 21474841126 00000004:00080000:0.0:1769074762.368515:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ded80, transno 21474841127 00000004:00080000:0.0:1769074762.368517:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ef800, transno 21474841128 00000004:00080000:0.0:1769074762.368520:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc700, transno 21474841129 00000004:00080000:0.0:1769074762.368523:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ec380, transno 21474841130 00000004:00080000:0.0:1769074762.368525:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddc00, transno 21474841131 00000004:00080000:0.0:1769074762.368527:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef5180, transno 21474841132 00000004:00080000:0.0:1769074762.368530:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dd880, transno 21474841133 00000004:00080000:0.0:1769074762.368533:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef6680, transno 21474841134 00000100:00100000:0.0:1769074762.368539:0:3650:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b54759f480 x1855007625081984/t21474841156(21474841156) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074778 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1769074762.368549:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13773 00000004:00080000:0.0:1769074762.368553:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759f480/2, rc 0, transno 21474841156 00000100:00100000:0.0:1769074762.368561:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625081984:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074762.368564:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625083008:12345-0@lo:2:osp-syn-1-0.0 Request processed in 100us (207us total) trans 17179874470 rc 0/0 00000100:00100000:3.0:1769074762.368571:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13778 00000100:00100000:1.0:1769074762.368573:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625083520 00000100:00100000:1.0:1769074762.368578:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625083520 00000100:00100000:1.0:1769074762.368608:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13779 00000100:00100000:1.0:1769074762.368616:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625083520:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074762.368688:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074762.368692:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.368696:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.368698:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.368700:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074762.368730:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625083520:12345-0@lo:2:osp-syn-1-0.0 Request processed in 115us (306us total) trans 17179874471 rc 0/0 00000100:00100000:1.0:1769074762.368736:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13779 00000100:00080000:1.0:1769074762.368765:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074762.368777:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074762.368888:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625082112 00000100:00100000:3.0:1769074762.368893:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625082112 00000100:00100000:3.0:1769074762.368917:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13780 00000100:00100000:3.0:1769074762.368923:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625082112:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074762.369000:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625082112:12345-0@lo:2:osp-syn-0-0.0 Request processed in 76us (220us total) trans 21474841160 rc 0/0 00000100:00100000:3.0:1769074762.369005:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13780 00000004:00080000:1.0:1769074762.369122:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759d500/2, rc 0, transno 21474841160 00000100:00100000:1.0:1769074762.369129:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625082112:0@lo:2:osp-syn-0-0.0 00000004:00010000:3.0:1769074762.369132:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000073000c5b reply state 00000000540b154e transno 8589943861 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074762.369146:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625083136:0@lo:2:osp-syn-1-0.0 00010000:00010000:3.0:1769074762.369154:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.369175:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00000100:00100000:1.0:1769074762.369176:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1769074762.369181:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:3.0:1769074762.369186:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00080000:3.0:1769074762.369265:0:6512:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:1.0:1769074762.369275:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625083648:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074762.369282:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62631:x1855007616719360:12345-192.168.204.60@tcp:36:mirror_io.0 Request processed in 32819us (32996us total) trans 8589943861 rc 0/0 00000100:00100000:3.0:1769074762.369290:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13256 00000100:00100000:3.0:1769074762.369314:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625083136 00000100:00100000:3.0:1769074762.369319:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625083136 00000100:00100000:3.0:1769074762.369343:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13781 00000100:00100000:3.0:1769074762.369348:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625083136:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.369374:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.369399:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1769074762.369407:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625083264:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.370840:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.370883:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625083776:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.370910:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.370949:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625083264 00000100:00100000:1.0:1769074762.370957:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625083264 00000100:00100000:1.0:1769074762.370995:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625083776 00000100:00100000:1.0:1769074762.370998:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625083776 00000100:00100000:1.0:1769074762.371023:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13782 00000100:00100000:1.0:1769074762.371030:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3651:x1855007625083264:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.371195:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3651:x1855007625083264:12345-0@lo:2:osp-syn-1-0.0 Request processed in 164us (356us total) trans 17179874473 rc 0/0 00000100:00100000:1.0:1769074762.371204:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13782 00000100:00100000:1.0:1769074762.371223:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13783 00000100:00100000:1.0:1769074762.371229:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3651:x1855007625083776:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074762.371283:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074762.371287:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.371289:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.371291:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.371293:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:1.0:1769074762.371318:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3651:x1855007625083776:12345-0@lo:2:osp-syn-1-0.0 Request processed in 89us (407us total) trans 17179874474 rc 0/0 00000100:00100000:1.0:1769074762.371324:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13783 00000100:00100000:1.0:1769074762.371351:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625083392:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.371376:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.371395:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625083904:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.371414:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1769074762.371450:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759df80/2, rc 0, transno 21474841153 00000100:00100000:1.0:1769074762.371457:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625082368:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074762.372373:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759b480/2, rc 0, transno 21474841154 00000100:00100000:1.0:1769074762.372381:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625082880:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074762.372424:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759fb80/2, rc 0, transno 21474841158 00000100:00100000:1.0:1769074762.372428:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625082624:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074762.372437:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759ce00/2, rc 0, transno 21474841155 00000100:00100000:1.0:1769074762.372440:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625082240:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074762.372450:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475de300/2, rc 0, transno 21474841157 00000100:00100000:1.0:1769074762.372454:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625082752:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.372615:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874457 gen 2 00000004:00080000:1.0:1769074762.372621:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df480, transno 17179874431 00000004:00080000:1.0:1769074762.372637:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddf80, transno 17179874432 00000004:00080000:1.0:1769074762.372639:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef4e00, transno 17179874433 00000004:00080000:1.0:1769074762.372642:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dca80, transno 17179874434 00000004:00080000:1.0:1769074762.372645:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dd500, transno 17179874435 00000004:00080000:1.0:1769074762.372647:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dfb80, transno 17179874436 00000004:00080000:1.0:1769074762.372650:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef5500, transno 17179874437 00000004:00080000:1.0:1769074762.372652:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc000, transno 17179874438 00000004:00080000:1.0:1769074762.372655:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df800, transno 17179874439 00000004:00080000:1.0:1769074762.372657:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545108e00, transno 17179874440 00000004:00080000:1.0:1769074762.372660:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d6300, transno 17179874441 00000004:00080000:1.0:1769074762.372663:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54510bb80, transno 17179874442 00000004:00080000:1.0:1769074762.372666:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d5f80, transno 17179874443 00000004:00080000:1.0:1769074762.372669:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ece00, transno 17179874444 00000004:00080000:1.0:1769074762.372671:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54510ad80, transno 17179874445 00000004:00080000:1.0:1769074762.372673:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d4700, transno 17179874446 00000004:00080000:1.0:1769074762.372676:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491eea00, transno 17179874447 00000004:00080000:1.0:1769074762.372679:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54510aa00, transno 17179874448 00000004:00080000:1.0:1769074762.372682:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cde7100, transno 17179874449 00000004:00080000:1.0:1769074762.372684:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5437b4000, transno 17179874450 00000004:00080000:1.0:1769074762.372687:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a7b80, transno 17179874451 00000004:00080000:1.0:1769074762.372690:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5437b4700, transno 17179874452 00000004:00080000:1.0:1769074762.372693:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a6a00, transno 17179874453 00000004:00080000:1.0:1769074762.372696:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5437b6a00, transno 17179874454 00000004:00080000:1.0:1769074762.372699:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cde4a80, transno 17179874455 00000004:00080000:1.0:1769074762.372703:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a6300, transno 17179874456 00000004:00080000:1.0:1769074762.372707:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fa20000, transno 17179874457 00000100:00100000:1.0:1769074762.372715:0:3651:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b545389f80 x1855007625083264/t17179874473(17179874473) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074778 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1769074762.372726:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545389f80/2, rc 0, transno 17179874473 00000100:00100000:1.0:1769074762.372737:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625083264:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074762.372756:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545389500/2, rc 0, transno 17179874474 00000100:00100000:1.0:1769074762.372761:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625083776:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074762.373101:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 8 in progress, 6 in flight, 27 done 00000100:00100000:3.0:1769074762.373737:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625083136:12345-0@lo:2:osp-syn-1-0.0 Request processed in 4383us (4557us total) trans 17179874472 rc 0/0 00000100:00100000:3.0:1769074762.373747:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13781 00000100:00100000:3.0:1769074762.373772:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625084032 00000100:00100000:3.0:1769074762.373779:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625084032 00000100:00100000:3.0:1769074762.374059:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13787 00000100:00100000:3.0:1769074762.374076:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625084032:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074762.374413:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:3.0:1769074762.374419:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:3.0:1769074762.374422:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769074762.374424:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:3.0:1769074762.374427:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:3.0:1769074762.374509:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625084032:12345-0@lo:2:osp-syn-1-0.0 Request processed in 440us (1234us total) trans 17179874478 rc 0/0 00000100:00100000:3.0:1769074762.374515:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13787 00000004:00080000:1.0:1769074762.375156:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 8 in progress, 1 in flight, 24 done 00000004:00080000:3.0:1769074762.377073:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475d9c00/2, rc 0, transno 17179874478 00000100:00100000:3.0:1769074762.377087:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625084032:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.377286:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759e680/2, rc 0, transno 21474841159 00000100:00100000:0.0:1769074762.377293:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625082496:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074762.377312:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545388380/2, rc 0, transno 17179874470 00000100:00100000:0.0:1769074762.377317:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625083008:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.377325:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545388000/2, rc 0, transno 17179874471 00000100:00100000:0.0:1769074762.377328:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625083520:0@lo:2:osp-syn-1-0.0 00010000:00010000:0.0:1769074762.377349:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.377359:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.377367:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.377373:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074762.377377:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.377385:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.377399:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.377402:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074762.377406:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074762.377412:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd52d49 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:0.0:1769074762.383567:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.383677:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720512 00000100:00100000:3.0:1769074762.383688:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720512 00000100:00100000:3.0:1769074762.383748:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13257 00000100:00100000:3.0:1769074762.383758:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62631:x1855007616720512:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074762.383782:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.383834:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b36a expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.383904:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.383918:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.383937:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.383941:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074762.383944:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f200, rc 0) 00000100:00100000:3.0:1769074762.384009:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62631:x1855007616720512:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 252us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.384016:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13257 00000100:00100000:1.0:1769074762.394004:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.394549:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720640 00000100:00100000:0.0:1769074762.394560:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720640 00000100:00100000:0.0:1769074762.394635:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 987 00000100:00100000:0.0:1769074762.394645:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62631:x1855007616720640:12345-192.168.204.60@tcp:103:mirror_io.0 00010000:00010000:0.0:1769074762.394664:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.394690:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54cf6d800/0x7fe2323adfd52d11 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b363 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.394725:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.394730:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.394800:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62631:x1855007616720640:12345-192.168.204.60@tcp:103:mirror_io.0 Request processed in 156us (808us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.394866:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 987 00000100:00100000:1.0:1769074762.402997:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616720896 00000100:00100000:1.0:1769074762.403009:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616720896 00000100:00100000:1.0:1769074762.403120:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4627 00000100:00100000:1.0:1769074762.403132:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62631:x1855007616720896:12345-192.168.204.60@tcp:35:mirror_io.0 00000100:00100000:1.0:1769074762.403399:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62631:x1855007616720896:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 269us (484us total) trans 8589943862 rc 0/0 00000100:00100000:1.0:1769074762.403407:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4627 00000100:00100000:0.0:1769074762.405912:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.405969:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616721152 00000100:00100000:1.0:1769074762.405983:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616721152 00000100:00100000:1.0:1769074762.406089:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4628 00000100:00100000:1.0:1769074762.406100:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62631:x1855007616721152:12345-192.168.204.60@tcp:35:mirror_io.0 00000100:00100000:1.0:1769074762.406613:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62631:x1855007616721152:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 512us (707us total) trans 8589943863 rc 0/0 00000100:00100000:1.0:1769074762.406622:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4628 00000100:00100000:3.0:1769074762.463876:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616721408 00000100:00100000:3.0:1769074762.463887:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616721408 00000100:00100000:3.0:1769074762.464013:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13258 00000100:00100000:3.0:1769074762.464023:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+7:62642:x1855007616721408:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.464049:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.464075:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52d57 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b371 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.464149:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464163:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464175:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464190:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464209:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464215:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464331:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464340:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464346:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074762.464356:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074762.464436:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464456:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464462:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464469:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464475:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074762.464487:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464495:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464508:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074762.464511:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074762.464516:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464522:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd52d5e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464535:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464542:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464547:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464552:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074762.464555:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464561:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464568:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ea200) 00010000:00010000:3.0:1769074762.464571:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074762.464573:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464578:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52d65 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074762.464620:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52d57 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.464633:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.464640:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.464642:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074762.464645:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:3.0:1769074762.464719:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62642:x1855007616721408:12345-192.168.204.60@tcp:101:cat.0 Request processed in 697us (967us total) trans 8589943864 rc 0/0 00000100:00100000:3.0:1769074762.464728:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13258 00000100:00100000:3.0:1769074762.479940:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.479993:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.480027:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.480147:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616721664 00000100:00100000:1.0:1769074762.480152:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616721664 00000100:00100000:1.0:1769074762.480192:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13259 00000100:00100000:1.0:1769074762.480215:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+220:62643:x1855007616721664:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074762.480242:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074762.480269:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52d7a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b378 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.480373:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480390:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480404:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480420:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480427:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480433:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480584:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480594:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480601:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074762.480615:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074762.480697:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480704:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480710:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480716:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480722:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074762.480726:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480732:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480744:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074762.480747:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074762.480752:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480758:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52d81 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480771:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480777:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480785:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480789:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074762.480792:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480798:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480804:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294700) 00010000:00010000:1.0:1769074762.480847:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074762.480851:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480856:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52d88 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.480897:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52d7a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 223 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.480908:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.480915:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.480918:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074762.480920:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:1.0:1769074762.480975:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62643:x1855007616721664:12345-192.168.204.60@tcp:101:cat.0 Request processed in 763us (1081us total) trans 8589943865 rc 0/0 00000100:00100000:1.0:1769074762.480983:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13259 00000100:00100000:3.0:1769074762.486063:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.494743:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616721920 00000100:00100000:3.0:1769074762.494762:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616721920 00000100:00100000:3.0:1769074762.495040:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13791 00000100:00100000:3.0:1769074762.495050:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616721920:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.495068:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.495089:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b386 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074762.495112:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2ba:0x0] initial LVB size: 2097152, mtime: 0x6971f049, atime: 0x6971f048, ctime: 0x6971f049, blocks: 0x800 00010000:00010000:3.0:1769074762.495132:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.495134:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.495143:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b386 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.495160:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b386 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.495173:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f600, rc 0) 00000100:00100000:3.0:1769074762.495240:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616721920:12345-192.168.204.60@tcp:101:cat.0 Request processed in 192us (9190us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.495247:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13791 00000100:00100000:3.0:1769074762.495693:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.496225:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.496251:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616722176 00000100:00100000:3.0:1769074762.496259:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616722176 00000100:00100000:3.0:1769074762.496289:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13792 00000100:00100000:3.0:1769074762.496297:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616722176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.496310:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.496325:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b394 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074762.496342:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2be:0x0] initial LVB size: 2097152, mtime: 0x6971f048, atime: 0x6971f046, ctime: 0x6971f048, blocks: 0x808 00010000:00010000:3.0:1769074762.496355:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.496357:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.496364:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.496378:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.496388:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:3.0:1769074762.496420:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616722176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 123us (197us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.496427:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13792 00000100:00100000:3.0:1769074762.497405:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.497428:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616722432 00000100:00100000:3.0:1769074762.497434:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616722432 00000100:00100000:3.0:1769074762.497461:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13793 00000100:00100000:3.0:1769074762.497468:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616722432:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.497489:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.497504:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3a2 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074762.497522:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bc:0x0] initial LVB size: 4194304, mtime: 0x6971f049, atime: 0x6971f049, ctime: 0x6971f049, blocks: 0x1008 00010000:00010000:3.0:1769074762.497536:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.497538:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.497545:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.497558:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.497567:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f800, rc 0) 00000100:00100000:3.0:1769074762.497599:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616722432:12345-192.168.204.60@tcp:101:cat.0 Request processed in 132us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.497608:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13793 00000100:00100000:3.0:1769074762.497642:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.497716:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.497735:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616722560 00000100:00100000:3.0:1769074762.497739:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616722560 00000100:00100000:3.0:1769074762.497757:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616722688 00000100:00100000:3.0:1769074762.497760:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616722688 00000100:00100000:3.0:1769074762.497778:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13794 00000100:00100000:3.0:1769074762.497784:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616722560:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.497792:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.497802:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3a9 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.497833:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.497835:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.497840:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.497852:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.497861:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:3.0:1769074762.497999:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616722560:12345-192.168.204.60@tcp:101:cat.0 Request processed in 213us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.498006:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13794 00000100:00100000:3.0:1769074762.498019:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13795 00000100:00100000:3.0:1769074762.498025:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616722688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.498032:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.498040:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3b0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.498052:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498053:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498058:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b0 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498069:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b0 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498078:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:3.0:1769074762.498094:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616722688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 69us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.498100:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13795 00000100:00100000:3.0:1769074762.498131:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.498160:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.498173:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616722816 00000100:00100000:3.0:1769074762.498177:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616722816 00000100:00100000:3.0:1769074762.498193:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616722944 00000100:00100000:3.0:1769074762.498196:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616722944 00000100:00100000:3.0:1769074762.498209:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13796 00000100:00100000:3.0:1769074762.498215:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616722816:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.498221:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.498228:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3b7 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.498244:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498245:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498250:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498260:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498268:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:3.0:1769074762.498286:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616722816:12345-192.168.204.60@tcp:101:cat.0 Request processed in 71us (155us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.498292:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13796 00000100:00100000:3.0:1769074762.498304:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13797 00000100:00100000:3.0:1769074762.498310:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616722944:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.498316:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.498322:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3be expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.498333:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498334:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498339:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498349:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498363:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e000, rc 0) 00000100:00100000:3.0:1769074762.498378:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616722944:12345-192.168.204.60@tcp:101:cat.0 Request processed in 69us (218us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.498384:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13797 00000100:00100000:3.0:1769074762.498411:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.498431:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.498440:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616723072 00000100:00100000:3.0:1769074762.498444:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616723072 00000100:00100000:3.0:1769074762.498457:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616723200 00000100:00100000:3.0:1769074762.498459:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616723200 00000100:00100000:3.0:1769074762.498468:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13798 00000100:00100000:3.0:1769074762.498473:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616723072:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.498490:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.498498:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3c5 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.498509:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498511:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498515:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498524:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 21 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498533:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ec00, rc 0) 00000100:00100000:3.0:1769074762.498550:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616723072:12345-192.168.204.60@tcp:101:cat.0 Request processed in 76us (138us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.498556:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13798 00000100:00100000:3.0:1769074762.498567:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13799 00000100:00100000:3.0:1769074762.498573:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616723200:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.498582:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.498589:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3cc expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.498721:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498722:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.498727:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 24 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498737:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 24 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.498745:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e400, rc 0) 00000100:00100000:3.0:1769074762.498766:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616723200:12345-192.168.204.60@tcp:101:cat.0 Request processed in 193us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.498838:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13799 00000100:00100000:1.0:1769074762.499162:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616723456 00000100:00100000:1.0:1769074762.499169:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616723456 00000100:00100000:1.0:1769074762.499212:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13260 00000100:00100000:1.0:1769074762.499219:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62640:x1855007616723456:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074762.499234:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074762.499248:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52df1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3d3 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.499355:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.499372:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.499382:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 223 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.499390:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 5/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.499419:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 223 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.499432:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 223 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.499446:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084160:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.499524:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.499538:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 8 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.499547:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084288:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.499571:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074762.514245:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.514364:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616723712 00000100:00100000:0.0:1769074762.514376:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616723712 00000100:00100000:0.0:1769074762.514459:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13261 00000100:00100000:0.0:1769074762.514470:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62641:x1855007616723712:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074762.514506:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.514531:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52dff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3da expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.514665:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074762.514682:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074762.514694:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000100:00100000:3.0:1769074762.521932:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.522995:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084160:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.523446:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616723840 00000100:00100000:0.0:1769074762.523458:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616723840 00010000:00010000:0.0:1769074762.523517:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.523576:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6842 00000100:00100000:0.0:1769074762.523587:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2408:x1855007616723840:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.523604:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.523612:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.523626:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.523654:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52d8f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b378 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.523678:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.523685:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.523696:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.523703:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.533874:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2408:x1855007616723840:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10275us (12673us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.533889:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6842 00000100:00100000:0.0:1769074762.534555:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616723968 00000100:00100000:0.0:1769074762.534565:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616723968 00010000:00010000:0.0:1769074762.534602:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 11 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.534634:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6843 00000100:00100000:0.0:1769074762.534643:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616723968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.534656:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.534661:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 11 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.534672:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 11 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.534695:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.534699:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.534709:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.534717:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.534724:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074762.534728:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074762.534733:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074762.534761:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.534804:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616723968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 162us (422us total) trans 0 rc 0/0 00010000:00010000:1.0:1769074762.535630:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074762.535641:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074762.535663:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074762.535847:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd52dff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 13 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.535866:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 12 pid: 8855 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.536971:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6843 00000100:00100000:0.0:1769074762.537004:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724096 00000100:00100000:0.0:1769074762.537009:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724096 00000100:00100000:0.0:1769074762.537043:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13262 00000100:00100000:0.0:1769074762.537051:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62643:x1855007616724096:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074762.537074:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.537103:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3e1 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.537166:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 224 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.537182:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 224 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.537320:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.537325:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074762.537332:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8200, rc 0) 00000100:00100000:1.0:1769074762.537417:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62641:x1855007616723712:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 22945us (23181us total) trans 8589943866 rc 0/0 00000100:00100000:1.0:1769074762.537427:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13261 00010000:00010000:3.0:1769074762.537634:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52d6c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b371 expref: 10 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.537656:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084288:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.537693:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.537703:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.537708:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074762.537724:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074762.538439:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.538444:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074762.538448:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9600, rc 0) 00000100:00100000:0.0:1769074762.538507:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:62643:x1855007616724096:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1454us (3536us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.538515:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13262 00010000:00010000:3.0:1769074762.538641:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52df1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 225 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.538665:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 224 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.538671:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.538674:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1769074762.538676:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164000, rc 0) 00000100:00100000:3.0:1769074762.538702:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:62640:x1855007616723456:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 39483us (43012us total) trans 8589943867 rc 0/0 00000100:00100000:3.0:1769074762.538709:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13260 00000100:00100000:3.0:1769074762.538792:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.540064:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.540221:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724352 00000100:00100000:0.0:1769074762.540231:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724352 00002000:00100000:0.0:1769074762.540273:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547ef7b80 x1855007616724352/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074762.540284:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7b80 with x1855007616724352 ext(1048576->2097151) 00002000:00010000:0.0:1769074762.540289:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef7b80 00000100:00100000:0.0:1769074762.540324:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8349 00000100:00100000:0.0:1769074762.540332:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616724352:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074762.559284:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.565993:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724480 00000100:00100000:1.0:1769074762.566011:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724480 00002000:00100000:1.0:1769074762.566073:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b547ef7800 x1855007616724480/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.566085:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7800 with x1855007616724480 ext(1048576->2097151) 00002000:00010000:1.0:1769074762.566092:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef7800 00000100:00100000:1.0:1769074762.566120:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724608 00000100:00100000:1.0:1769074762.566125:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724608 00002000:00100000:1.0:1769074762.566141:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54538ad80 x1855007616724608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.566148:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54538ad80 with x1855007616724608 ext(1048576->2097151) 00002000:00010000:1.0:1769074762.566151:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54538ad80 00000100:00100000:1.0:1769074762.566173:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8350 00000100:00100000:1.0:1769074762.566182:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616724480:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074762.567096:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8351 00000100:00100000:0.0:1769074762.567116:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616724608:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074762.584917:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616724224:12345-192.168.204.60@tcp:3:cat.0 Request processed in 46942us (48248us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074762.584932:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8348 00002000:00100000:3.0:1769074762.584946:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54d0e4700 x1855007616724224/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.584965:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4700 with x1855007616724224 ext(0->1048575) 00002000:00010000:3.0:1769074762.584971:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e4700 00000100:00100000:1.0:1769074762.606930:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616724480:12345-192.168.204.60@tcp:3:cat.0 Request processed in 40732us (66856us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074762.606947:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8350 00002000:00100000:1.0:1769074762.606962:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b547ef7800 x1855007616724480/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.606977:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7800 with x1855007616724480 ext(1048576->2097151) 00002000:00010000:1.0:1769074762.606988:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef7800 00000100:00100000:3.0:1769074762.610417:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616724352:12345-192.168.204.60@tcp:3:cat.0 Request processed in 70074us (71614us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074762.610431:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8349 00002000:00100000:3.0:1769074762.610443:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547ef7b80 x1855007616724352/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.610454:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7b80 with x1855007616724352 ext(1048576->2097151) 00002000:00010000:3.0:1769074762.610460:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef7b80 00000100:00100000:3.0:1769074762.629215:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724736 00000100:00100000:3.0:1769074762.629234:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724736 00000100:00100000:3.0:1769074762.630663:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13263 00000100:00100000:3.0:1769074762.630691:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62642:x1855007616724736:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.630730:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.630755:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e22 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3e8 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.636682:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.636753:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.636898:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724864 00000100:00100000:1.0:1769074762.636915:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724864 00000100:00100000:0.0:1769074762.636970:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00100000:1.0:1769074762.636995:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b549f2b100 x1855007616724864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.637009:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2b100 with x1855007616724864 ext(0->1048575) 00002000:00010000:1.0:1769074762.637018:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549f2b100 00000100:00100000:1.0:1769074762.637044:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616724992 00000100:00100000:1.0:1769074762.637047:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616724992 00002000:00100000:1.0:1769074762.637066:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b549f2a300 x1855007616724992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.637074:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2a300 with x1855007616724992 ext(1048576->2097151) 00002000:00010000:1.0:1769074762.637077:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f2a300 00000100:00100000:0.0:1769074762.637334:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616725120 00000100:00100000:0.0:1769074762.637340:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616725120 00002000:00100000:0.0:1769074762.637362:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b549f29500 x1855007616725120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074762.637370:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f29500 with x1855007616725120 ext(1048576->2097151) 00002000:00010000:0.0:1769074762.637374:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f29500 00000100:00100000:0.0:1769074762.638416:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8352 00000100:00100000:0.0:1769074762.638430:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616724864:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074762.639431:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8354 00000100:00100000:1.0:1769074762.639452:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616725120:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074762.662704:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616725248 00000100:00100000:1.0:1769074762.662729:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616725248 00002000:00100000:1.0:1769074762.662903:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54d0e7b80 x1855007616725248/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074762.662917:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e7b80 with x1855007616725248 ext(1048576->2097151) 00002000:00010000:1.0:1769074762.662925:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e7b80 00000100:00100000:1.0:1769074762.662976:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8355 00000100:00100000:1.0:1769074762.662993:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616725248:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074762.685599:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616724864:12345-192.168.204.60@tcp:3:cat.0 Request processed in 47161us (48926us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074762.685612:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8352 00002000:00100000:0.0:1769074762.685625:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b549f2b100 x1855007616724864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074762.685641:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2b100 with x1855007616724864 ext(0->1048575) 00002000:00010000:0.0:1769074762.685646:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549f2b100 00000100:00100000:3.0:1769074762.692391:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616725248:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29391us (33169us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074762.692406:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8355 00002000:00100000:3.0:1769074762.692418:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54d0e7b80 x1855007616725248/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.692429:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e7b80 with x1855007616725248 ext(1048576->2097151) 00002000:00010000:3.0:1769074762.692435:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e7b80 00000100:00100000:1.0:1769074762.768080:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616725504 00000100:00100000:1.0:1769074762.768091:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616725504 00000100:00100000:1.0:1769074762.768169:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13264 00000100:00100000:1.0:1769074762.768180:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62641:x1855007616725504:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074762.768207:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074762.768210:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074762.768239:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e22 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e8 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.768266:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.768270:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074762.768285:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89600/0x7fe2323adfd52e29 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3ef expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.768339:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.768354:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.768363:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.768372:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 5/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.768417:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.768430:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 223 pid: 10283 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.768443:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084416:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.768589:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.768599:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 9 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.768607:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084544:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.768635:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074762.775337:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616725632 00000100:00100000:0.0:1769074762.775349:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616725632 00010000:00010000:0.0:1769074762.775390:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 224 pid: 10283 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.775436:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616725760 00000100:00100000:0.0:1769074762.775439:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616725760 00010000:00010000:0.0:1769074762.775452:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 10 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.775484:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6844 00000100:00100000:0.0:1769074762.775494:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2408:x1855007616725632:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.775509:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.775516:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 224 pid: 10283 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.775528:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 224 pid: 10283 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.775549:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.775555:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.775563:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.775573:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.775634:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2408:x1855007616725632:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 141us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.775642:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6844 00000100:00100000:0.0:1769074762.775662:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6845 00000100:00100000:0.0:1769074762.775669:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616725760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.775677:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.775682:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 10 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.775691:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 10 pid: 6512 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.775704:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.775707:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.775715:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.775722:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.775739:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.775759:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616725760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 90us (511us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.775765:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6845 00010000:00010000:1.0:1769074762.776538:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd52e14 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3e1 expref: 223 pid: 10283 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.776559:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084416:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.776572:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd52d50 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b36a expref: 9 pid: 6512 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.776582:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625084544:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.776608:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.776616:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.776621:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000040:00080000:1.0:1769074762.777034:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2402, 88 off205592 00000004:00080000:1.0:1769074762.777043:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750619235612ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.777174:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2403, 88 off205416 00000004:00080000:1.0:1769074762.777180:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750619372589ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074762.777205:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2404, 88 off205504 00000004:00080000:1.0:1769074762.777209:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750619402075ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074762.777219:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2403, 88 off205680 00000004:00080000:1.0:1769074762.777223:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750619415886ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.781492:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2404, 88 off205768 00000004:00080000:1.0:1769074762.781512:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750623704678ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.781541:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2405, 88 off205592 00000004:00080000:1.0:1769074762.781545:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750623738181ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074762.781625:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2406, 88 off205680 00000004:00080000:1.0:1769074762.781631:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750623823351ns, next at 1706953247299ns, rc = 0 00000100:00100000:3.0:1769074762.785735:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616725888 00000100:00100000:3.0:1769074762.785748:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616725888 00002000:00100000:3.0:1769074762.785953:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54d0e5c00 x1855007616725888/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.785965:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5c00 with x1855007616725888 ext(2097152->4194303) 00002000:00010000:3.0:1769074762.785972:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e5c00 00000100:00100000:3.0:1769074762.785998:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616726016 00000100:00100000:3.0:1769074762.786002:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616726016 00002000:00100000:3.0:1769074762.786020:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d0e5f80 x1855007616726016/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.786029:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5f80 with x1855007616726016 ext(2097152->4194303) 00002000:00010000:3.0:1769074762.786032:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e5f80 00000100:00100000:3.0:1769074762.786050:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8356 00000100:00100000:3.0:1769074762.786058:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616725888:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074762.790929:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000040:00080000:1.0:1769074762.791448:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2407, 88 off205768 00000004:00080000:1.0:1769074762.791462:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750633654633ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074762.791495:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2405, 88 off205856 00000004:00080000:1.0:1769074762.791499:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750633692242ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.791521:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2408, 88 off205856 00000004:00080000:1.0:1769074762.791525:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750633718163ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074762.791535:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2406, 88 off205944 00000004:00080000:1.0:1769074762.791539:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750633732118ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.791548:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2409, 88 off205944 00000004:00080000:1.0:1769074762.791553:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750633745772ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074762.791561:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2407, 88 off206032 00000004:00080000:1.0:1769074762.791565:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750633758302ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.791574:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2408, 88 off206120 00000004:00080000:1.0:1769074762.791578:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750633771488ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.791588:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2409, 88 off206208 00000004:00080000:1.0:1769074762.791591:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750633784322ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.791600:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2410, 88 off206296 00000004:00080000:1.0:1769074762.791603:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1750633796445ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074762.791612:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2410, 88 off206032 00000004:00080000:1.0:1769074762.791615:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1750633808223ns, next at 1706953247299ns, rc = 0 00000001:00080000:1.0:1769074762.797994:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943868 is committed 00000004:00080000:1.0:1769074762.798007:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1914 committed 00000004:00080000:1.0:1769074762.798012:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1913 committed 00000004:00080000:1.0:1769074762.798024:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1912 committed 00000004:00080000:1.0:1769074762.798028:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1911 committed 00000004:00080000:1.0:1769074762.798030:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1910 committed 00000004:00080000:1.0:1769074762.798033:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1913 committed 00000004:00080000:1.0:1769074762.798035:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1909 committed 00000004:00080000:1.0:1769074762.798038:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1912 committed 00000004:00080000:1.0:1769074762.798040:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1908 committed 00000004:00080000:1.0:1769074762.798042:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1911 committed 00000004:00080000:1.0:1769074762.798044:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1910 committed 00000004:00080000:1.0:1769074762.798046:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1909 committed 00000004:00080000:1.0:1769074762.798048:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1907 committed 00000004:00080000:1.0:1769074762.798050:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1906 committed 00000004:00080000:1.0:1769074762.798052:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1908 committed 00000004:00080000:1.0:1769074762.798054:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1907 committed 00000004:00080000:1.0:1769074762.798056:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1905 committed 00000001:00080000:1.0:1769074762.798061:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943867 is committed 00000001:00080000:1.0:1769074762.798074:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943863 is committed 00000100:00100000:0.0:1769074762.798425:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085056:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.798505:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.798548:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1769074762.798555:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625084928:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.798576:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.798593:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085440:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.798612:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.798716:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085056 00000100:00100000:0.0:1769074762.798724:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085056 00000100:00100000:0.0:1769074762.798766:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625084928 00000100:00100000:0.0:1769074762.798769:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625084928 00000100:00100000:0.0:1769074762.798784:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085440 00000100:00100000:0.0:1769074762.798787:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085440 00000100:00100000:0.0:1769074762.798827:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13800 00000100:00100000:0.0:1769074762.798836:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+16:3650:x1855007625085056:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074762.798979:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:0.0:1769074762.798983:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.798986:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.798988:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.798990:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074762.799030:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+17:3650:x1855007625085056:12345-0@lo:2:osp-syn-1-0.0 Request processed in 195us (524us total) trans 17179874479 rc 0/0 00000100:00100000:0.0:1769074762.799038:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13800 00000100:00100000:0.0:1769074762.799053:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13801 00000100:00100000:0.0:1769074762.799059:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+16:3650:x1855007625084928:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074762.799112:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074762.799116:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.799119:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime from disk: 1769074759 -> 1769074759 00002000:00010000:0.0:1769074762.799121:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.799123:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb blocks from disk: 5296 -> 5296 00000100:00100000:0.0:1769074762.799153:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+17:3650:x1855007625084928:12345-0@lo:2:osp-syn-1-0.0 Request processed in 94us (575us total) trans 17179874480 rc 0/0 00000100:00100000:1.0:1769074762.799157:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625084800:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.799164:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13801 00000100:00100000:0.0:1769074762.799179:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13802 00000100:00100000:0.0:1769074762.799187:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+16:3650:x1855007625085440:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.799198:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.799233:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085312:0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074762.799241:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074762.799244:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.799247:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.799249:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.799251:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:0.0:1769074762.799288:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+17:3650:x1855007625085440:12345-0@lo:2:osp-syn-1-0.0 Request processed in 102us (674us total) trans 17179874481 rc 0/0 00000100:00100000:1.0:1769074762.799296:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.799301:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13802 00000100:00100000:0.0:1769074762.799316:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625084800 00000100:00100000:0.0:1769074762.799320:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625084800 00000100:00100000:1.0:1769074762.799327:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1769074762.799337:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625084672:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.799344:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085312 00000100:00100000:0.0:1769074762.799348:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085312 00000100:00100000:1.0:1769074762.799369:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.799373:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13803 00000100:00100000:0.0:1769074762.799381:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+18:3652:x1855007625084800:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.799394:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085184:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.799427:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:0.0:1769074762.799440:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074762.799444:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074760 -> 1769074760 00002000:00010000:0.0:1769074762.799447:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074758 -> 1769074758 00002000:00010000:0.0:1769074762.799448:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074760 -> 1769074760 00002000:00010000:0.0:1769074762.799452:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:0.0:1769074762.799496:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+19:3652:x1855007625084800:12345-0@lo:2:osp-syn-1-0.0 Request processed in 117us (299us total) trans 17179874482 rc 0/0 00000100:00100000:0.0:1769074762.799502:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13803 00000100:00100000:0.0:1769074762.799510:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085184 00000100:00100000:0.0:1769074762.799513:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085184 00000100:00100000:0.0:1769074762.799524:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13804 00000100:00100000:0.0:1769074762.799528:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+19:3652:x1855007625085312:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074762.799560:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625084672 00000100:00100000:1.0:1769074762.799568:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625084672 00002000:00010000:0.0:1769074762.799574:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074762.799577:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.799580:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.799581:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.799584:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074762.799598:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13805 00000100:00100000:1.0:1769074762.799609:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b358c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+21:3652:x1855007625085184:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.799612:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+21:3652:x1855007625085312:12345-0@lo:2:osp-syn-1-0.0 Request processed in 81us (320us total) trans 17179874483 rc 0/0 00000100:00100000:0.0:1769074762.799622:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13804 00000100:00100000:0.0:1769074762.799637:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13806 00000100:00100000:0.0:1769074762.799644:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+20:3652:x1855007625084672:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074762.799699:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074762.799704:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.799707:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.799709:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:1.0:1769074762.799711:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074762.799714:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074756 -> 1769074756 00002000:00010000:1.0:1769074762.799716:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00002000:00010000:0.0:1769074762.799718:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime from disk: 1769074750 -> 1769074750 00002000:00010000:0.0:1769074762.799723:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074756 -> 1769074756 00002000:00010000:0.0:1769074762.799726:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb blocks from disk: 2024 -> 2024 00000100:00100000:1.0:1769074762.799761:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b358c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+22:3652:x1855007625085184:12345-0@lo:2:osp-syn-1-0.0 Request processed in 153us (332us total) trans 17179874484 rc 0/0 00000100:00100000:0.0:1769074762.799767:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+22:3652:x1855007625084672:12345-0@lo:2:osp-syn-1-0.0 Request processed in 122us (398us total) trans 17179874485 rc 0/0 00000100:00100000:1.0:1769074762.799774:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13805 00000100:00100000:0.0:1769074762.799782:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13806 00000100:00100000:0.0:1769074762.799921:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085568:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.799944:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1769074762.799979:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dd880/2, rc 0, transno 17179874480 00000100:00100000:0.0:1769074762.799990:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625084928:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.800000:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dd500/2, rc 0, transno 17179874481 00000100:00100000:0.0:1769074762.800003:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085440:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.800012:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475df480/2, rc 0, transno 17179874479 00000100:00100000:0.0:1769074762.800016:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085056:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.800033:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085568 00000100:00100000:0.0:1769074762.800037:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085568 00000100:00100000:0.0:1769074762.800060:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13807 00000100:00100000:0.0:1769074762.800066:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+21:3650:x1855007625085568:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074762.800118:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb size from disk: 8388608 -> 8388608 00002000:00010000:0.0:1769074762.800121:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.800123:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.800126:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.800128:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb blocks from disk: 8192 -> 8192 00000100:00100000:0.0:1769074762.800157:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+22:3650:x1855007625085568:12345-0@lo:2:osp-syn-1-0.0 Request processed in 91us (212us total) trans 17179874486 rc 0/0 00000100:00100000:0.0:1769074762.800163:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13807 00000004:00080000:0.0:1769074762.800189:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dfb80/2, rc 0, transno 17179874486 00000100:00100000:0.0:1769074762.800194:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085568:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.800236:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddc00/2, rc 0, transno 17179874485 00000100:00100000:0.0:1769074762.800239:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625084672:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.800247:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddf80/2, rc 0, transno 17179874484 00000100:00100000:0.0:1769074762.800251:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085184:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.800259:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc700/2, rc 0, transno 17179874482 00000100:00100000:0.0:1769074762.800262:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625084800:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074762.800269:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dca80/2, rc 0, transno 17179874483 00000100:00100000:0.0:1769074762.800272:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085312:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.800278:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074762.800283:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085696:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.800305:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.800327:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085696 00000100:00100000:0.0:1769074762.800331:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085696 00000100:00100000:0.0:1769074762.800348:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13808 00000100:00100000:0.0:1769074762.800354:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+22:3652:x1855007625085696:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074762.800405:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074762.800408:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.800410:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074762.800412:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074762.800414:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074762.800443:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+23:3652:x1855007625085696:12345-0@lo:2:osp-syn-1-0.0 Request processed in 89us (137us total) trans 17179874487 rc 0/0 00000100:00100000:0.0:1769074762.800450:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13808 00000004:00080000:0.0:1769074762.800487:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5446c2d80/2, rc 0, transno 17179874487 00000100:00100000:0.0:1769074762.800492:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085696:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074762.800535:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085824:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.800557:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.800579:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085824 00000100:00100000:0.0:1769074762.800584:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085824 00000100:00100000:0.0:1769074762.800610:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13809 00000100:00100000:0.0:1769074762.800616:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625085824:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074762.800685:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074762.800688:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.800690:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074760 -> 1769074760 00002000:00010000:0.0:1769074762.800692:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.800694:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074762.800722:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625085824:12345-0@lo:2:osp-syn-0-0.0 Request processed in 105us (163us total) trans 21474841161 rc 0/0 00000100:00100000:0.0:1769074762.800728:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13809 00000004:00080000:0.0:1769074762.800752:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5446c0e00/2, rc 0, transno 21474841161 00000100:00100000:0.0:1769074762.800756:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625085824:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.800786:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074762.800792:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085952:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.800835:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.800856:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625085952 00000100:00100000:0.0:1769074762.800859:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625085952 00000100:00100000:0.0:1769074762.800882:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13810 00000100:00100000:0.0:1769074762.800887:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625085952:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074762.800935:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074762.800939:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074760 -> 1769074760 00002000:00010000:0.0:1769074762.800941:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.800943:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074760 -> 1769074760 00002000:00010000:0.0:1769074762.800945:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074762.800972:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+15:3650:x1855007625085952:12345-0@lo:2:osp-syn-0-0.0 Request processed in 84us (136us total) trans 21474841162 rc 0/0 00000100:00100000:0.0:1769074762.800979:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13810 00000004:00080000:0.0:1769074762.801002:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5446c0000/2, rc 0, transno 21474841162 00000100:00100000:0.0:1769074762.801007:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625085952:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.801059:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625086080:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.801084:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1769074762.801089:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.801103:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.801107:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00000100:00100000:0.0:1769074762.801112:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625086080 00000100:00100000:0.0:1769074762.801117:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625086080 00010000:00010000:1.0:1769074762.801125:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89600/0x7fe2323adfd52e29 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 10 pid: 6514 timeout: 0 lvb_type: 3 00000100:00100000:0.0:1769074762.801145:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13811 00010000:00010000:1.0:1769074762.801151:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074762.801153:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+15:3650:x1855007625086080:12345-0@lo:2:osp-syn-0-0.0 00010000:00010000:1.0:1769074762.801186:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.801189:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:1.0:1769074762.801192:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00002000:00010000:0.0:1769074762.801213:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074762.801217:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.801220:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.801222:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074761 -> 1769074761 00002000:00010000:0.0:1769074762.801224:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:1.0:1769074762.801255:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62641:x1855007616725504:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 33075us (33295us total) trans 8589943868 rc 0/0 00000100:00100000:0.0:1769074762.801263:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+16:3650:x1855007625086080:12345-0@lo:2:osp-syn-0-0.0 Request processed in 107us (174us total) trans 21474841163 rc 0/0 00000100:00100000:1.0:1769074762.801270:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13264 00000100:00100000:0.0:1769074762.801279:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13811 00000004:00080000:0.0:1769074762.801310:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5446c2a00/2, rc 0, transno 21474841163 00000100:00100000:0.0:1769074762.801316:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625086080:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.801354:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074762.801362:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625086208:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.801394:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074762.801416:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625086336:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.801443:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074762.801868:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625086464:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.806446:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625086208 00000100:00100000:1.0:1769074762.806454:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625086208 00000100:00100000:1.0:1769074762.806507:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625086336 00000100:00100000:1.0:1769074762.806511:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625086336 00000100:00100000:1.0:1769074762.806532:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625086720 00000100:00100000:1.0:1769074762.806535:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625086720 00000100:00100000:1.0:1769074762.806555:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13812 00000100:00100000:1.0:1769074762.806563:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+18:3652:x1855007625086208:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074762.806709:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074762.806713:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.806715:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.806716:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.806718:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:1.0:1769074762.806803:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+19:3652:x1855007625086208:12345-0@lo:2:osp-syn-0-0.0 Request processed in 228us (5396us total) trans 21474841164 rc 0/0 00000100:00100000:1.0:1769074762.806828:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13812 00000100:00100000:1.0:1769074762.806849:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13813 00000100:00100000:1.0:1769074762.806856:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+18:3652:x1855007625086336:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074762.806905:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:1.0:1769074762.806908:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.806911:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.806913:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.806915:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074762.806943:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+19:3652:x1855007625086336:12345-0@lo:2:osp-syn-0-0.0 Request processed in 88us (5499us total) trans 21474841165 rc 0/0 00000100:00100000:1.0:1769074762.806949:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13813 00000100:00100000:1.0:1769074762.806962:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13814 00000100:00100000:1.0:1769074762.806968:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+18:3651:x1855007625086720:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074762.807020:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074762.807022:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074762.807024:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.807026:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074762.807029:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074762.807071:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+19:3651:x1855007625086720:12345-0@lo:2:osp-syn-0-0.0 Request processed in 103us (5511us total) trans 21474841166 rc 0/0 00000100:00100000:1.0:1769074762.807077:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13814 00000100:00100000:1.0:1769074762.807140:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625086592:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074762.807184:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.807224:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625086592 00000100:00100000:1.0:1769074762.807229:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625086592 00000100:00100000:1.0:1769074762.807585:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13815 00000100:00100000:1.0:1769074762.807592:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+19:3650:x1855007625086592:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074762.807667:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074762.807670:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074762.807672:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.807674:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074762.807676:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074762.807711:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+20:3650:x1855007625086592:12345-0@lo:2:osp-syn-0-0.0 Request processed in 120us (527us total) trans 21474841167 rc 0/0 00000100:00100000:1.0:1769074762.807717:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13815 00000004:00080000:1.0:1769074762.807765:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ed500/2, rc 0, transno 21474841167 00000100:00100000:1.0:1769074762.807772:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625086592:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074762.811004:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475df800/2, rc 0, transno 21474841165 00000100:00100000:0.0:1769074762.811019:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625086336:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074762.811033:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc000/2, rc 0, transno 21474841164 00000100:00100000:0.0:1769074762.811036:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625086208:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074762.811416:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491edf80/2, rc 0, transno 21474841166 00000100:00100000:0.0:1769074762.811422:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625086720:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074762.813970:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074762.815492:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625086464 00000100:00100000:1.0:1769074762.815512:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625086464 00000100:00100000:1.0:1769074762.815578:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13816 00000100:00100000:1.0:1769074762.815589:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+20:3649:x1855007625086464:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074762.815748:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074762.815753:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.815756:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074762.815758:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074762.815760:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074762.815889:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625086464:12345-0@lo:2:osp-syn-0-0.0 Request processed in 301us (1926us total) trans 21474841168 rc 0/0 00000100:00100000:1.0:1769074762.815896:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13816 00000004:00080000:3.0:1769074762.817237:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ded80/2, rc 0, transno 21474841168 00000100:00100000:3.0:1769074762.817249:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625086464:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074762.823167:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.823376:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616726400 00000100:00100000:0.0:1769074762.823387:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616726400 00000100:00100000:0.0:1769074762.823440:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13265 00000100:00100000:0.0:1769074762.823450:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:62640:x1855007616726400:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074762.823493:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.823515:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3f6 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.823576:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.823592:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.823612:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.823615:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074762.823618:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d96000, rc 0) 00000100:00100000:0.0:1769074762.823668:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62640:x1855007616726400:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 219us (511us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.823676:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13265 00000100:00100000:3.0:1769074762.823897:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616726016:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 37208us (38502us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074762.823906:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8357 00002000:00100000:3.0:1769074762.823919:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d0e5f80 x1855007616726016/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.823934:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5f80 with x1855007616726016 ext(2097152->4194303) 00002000:00010000:3.0:1769074762.823939:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e5f80 00000100:00100000:3.0:1769074762.860067:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616725888:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 73998us (74721us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074762.860081:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8356 00002000:00100000:3.0:1769074762.860094:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54d0e5c00 x1855007616725888/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.860103:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5c00 with x1855007616725888 ext(2097152->4194303) 00002000:00010000:3.0:1769074762.860108:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e5c00 00000100:00100000:3.0:1769074762.861076:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616726144:12345-192.168.204.60@tcp:3:cat.0 Request processed in 58114us (70152us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074762.861085:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8358 00002000:00100000:3.0:1769074762.861093:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b549f2b800 x1855007616726144/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.861102:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2b800 with x1855007616726144 ext(2097152->4194303) 00002000:00010000:3.0:1769074762.861107:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549f2b800 00000100:00100000:0.0:1769074762.885943:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616726272:12345-192.168.204.60@tcp:3:cat.0 Request processed in 80292us (81880us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074762.885957:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8359 00002000:00100000:0.0:1769074762.885971:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d0e6680 x1855007616726272/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074762.885980:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6680 with x1855007616726272 ext(2097152->4194303) 00002000:00010000:0.0:1769074762.885986:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e6680 00000100:00100000:1.0:1769074762.891260:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.891331:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.891485:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616726528 00000100:00100000:0.0:1769074762.891504:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616726528 00000100:00100000:0.0:1769074762.891588:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616726656 00000100:00100000:0.0:1769074762.891595:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616726656 00000100:00100000:0.0:1769074762.891652:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13266 00000100:00100000:0.0:1769074762.891673:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:62644:x1855007616726528:12345-192.168.204.60@tcp:101:lfs.0 00000100:00100000:3.0:1769074762.891681:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13267 00000100:00100000:3.0:1769074762.891696:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62643:x1855007616726656:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074762.891717:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.891718:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.891746:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e3e lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b404 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.891747:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52e45 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b3fd expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.891870:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e3e lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b404 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.891885:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.891888:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e3e lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b404 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.891913:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.891916:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.891922:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074762.891926:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00010000:00010000:0.0:1769074762.891931:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.891955:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.891965:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.891976:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074762.891988:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62643:x1855007616726656:12345-192.168.204.60@tcp:101:cat.0 Request processed in 295us (656us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.891997:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13267 00010000:00010000:0.0:1769074762.892085:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.892103:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.892116:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 8 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.892125:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 5/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.892169:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 8 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.892186:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 8 pid: 8855 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.892208:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625086848:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.892331:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.892346:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 222 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.892360:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625086976:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.892408:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074762.897458:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.897512:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.897572:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.897715:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616726784 00000100:00100000:3.0:1769074762.897724:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616726784 00000100:00100000:3.0:1769074762.897784:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616726912 00000100:00100000:3.0:1769074762.897788:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616726912 00010000:00010000:3.0:1769074762.897848:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 9 pid: 8855 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.897868:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616727040 00000100:00100000:3.0:1769074762.897871:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616727040 00010000:00010000:3.0:1769074762.897881:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074762.897897:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6846 00000100:00100000:3.0:1769074762.897906:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616726912:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.897921:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.897927:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 9 pid: 8855 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.897944:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 9 pid: 8855 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.897965:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.897970:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.897979:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.897985:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.897990:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.898044:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616726912:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.898052:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6846 00000100:00100000:3.0:1769074762.898070:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6847 00000100:00100000:3.0:1769074762.898077:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2407:x1855007616727040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.898084:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.898088:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.898098:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 224 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.898111:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.898115:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.898123:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.898130:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074762.898145:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.898167:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2407:x1855007616727040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 91us (595us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.898173:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6847 00000100:00100000:3.0:1769074762.898186:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 988 00000100:00100000:3.0:1769074762.898192:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491eea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:2409:x1855007616726784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074762.898199:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074762.898209:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e3e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b404 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.898223:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074762.898226:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074762.898245:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:2409:x1855007616726784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 52us (805us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.898251:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 988 00010000:00010000:0.0:1769074762.898841:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8200/0x7fe2323adfd52e06 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3da expref: 8 pid: 8855 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.898857:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625086848:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.898872:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd52df8 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3d3 expref: 221 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.898885:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625086976:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074762.898913:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.898923:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.898928:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074762.898990:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:0.0:1769074762.898996:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074762.899021:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899035:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899052:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899063:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899076:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899085:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899094:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074762.899102:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899115:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899136:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074762.899143:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074762.899150:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899156:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97800/0x7fe2323adfd52e4c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899169:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899174:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899179:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899184:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d96e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074762.899188:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899197:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899206:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47600) 00010000:00010000:0.0:1769074762.899208:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074762.899212:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899218:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd52e53 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.899276:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97400/0x7fe2323adfd52e45 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3fd expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.899297:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3fd expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.899305:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.899308:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074762.899311:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:0.0:1769074762.899644:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62644:x1855007616726528:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 7975us (8397us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.899660:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13266 00000100:00100000:1.0:1769074762.906506:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.906549:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.906584:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.906614:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.909220:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616727168 00000100:00100000:1.0:1769074762.909246:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616727168 00000100:00100000:1.0:1769074762.909320:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616727424 00000100:00100000:1.0:1769074762.909324:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616727424 00000100:00100000:3.0:1769074762.910013:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13817 00000100:00100000:3.0:1769074762.910027:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616727168:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074762.910047:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.910072:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b40b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.910112:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.910114:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.910128:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.910150:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.910167:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:3.0:1769074762.910408:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616727168:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 384us (3911us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.910421:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13817 00000100:00100000:3.0:1769074762.910556:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13818 00000100:00100000:3.0:1769074762.910565:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616727296:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074762.910574:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.910584:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b412 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.910596:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.910598:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.910603:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b412 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.910613:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b412 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.910626:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ea00, rc 0) 00000100:00100000:3.0:1769074762.910646:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616727296:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 84us (4097us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.910652:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13818 00000100:00100000:3.0:1769074762.910666:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13819 00000100:00100000:3.0:1769074762.910672:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616727424:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074762.910679:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.910685:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b419 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.910696:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.910698:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.910702:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.910711:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.910719:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e800, rc 0) 00000100:00100000:3.0:1769074762.910737:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616727424:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 65us (4152us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.910743:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13819 00000100:00100000:3.0:1769074762.910756:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13820 00000100:00100000:3.0:1769074762.910762:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616727552:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074762.910771:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.910779:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b420 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074762.911077:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.911105:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616727680 00000100:00100000:1.0:1769074762.911111:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616727680 00000100:00100000:1.0:1769074762.911137:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13821 00000100:00100000:1.0:1769074762.911143:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616727680:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074762.911151:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074762.911160:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b427 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.911171:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.911173:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.911179:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.911190:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.911198:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876400, rc 0) 00000100:00100000:1.0:1769074762.911219:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616727680:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 75us (144us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.911224:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13821 00010000:00010000:3.0:1769074762.911407:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.911409:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.911415:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.911428:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.911508:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f200, rc 0) 00000100:00100000:3.0:1769074762.911534:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616727552:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 772us (4920us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.911540:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13820 00000100:00100000:3.0:1769074762.911633:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.911649:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616727808 00000100:00100000:3.0:1769074762.911653:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616727808 00000100:00100000:3.0:1769074762.911677:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13822 00000100:00100000:3.0:1769074762.911684:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616727808:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074762.911691:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.911699:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b42e expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074762.911711:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.911713:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074762.911718:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.911727:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074762.911735:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:3.0:1769074762.911753:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616727808:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 70us (121us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074762.911759:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13822 00000100:00100000:0.0:1769074762.922770:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074762.922913:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.923487:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616727936 00000100:00100000:1.0:1769074762.923499:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616727936 00000100:00100000:1.0:1769074762.923616:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13823 00000100:00100000:1.0:1769074762.923636:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62641:x1855007616727936:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074762.923667:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074762.923700:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b435 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.923746:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 12 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.923765:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.923778:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.923787:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.923789:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074762.923794:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [667648->1048575] (req 667648->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.923868:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.923882:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 20 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.923895:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dfb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625087104:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.924020:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.924037:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 11 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.924052:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625087232:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.924113:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 12 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074762.924127:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 12 pid: 36516 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.924140:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625087360:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.924178:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074762.935091:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625087232:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.935106:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625087360:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074762.937882:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616728320 00000100:00100000:0.0:1769074762.937899:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616728320 00000100:00100000:0.0:1769074762.937999:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13824 00000100:00100000:0.0:1769074762.938011:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62640:x1855007616728320:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074762.938033:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074762.938056:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b443 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.938092:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.938099:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [667648->1048575] (req 667648->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.938106:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074762.938111:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [585728->1048575] (req 585728->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.938171:0:28955:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [667648->1048575] (req 667648->1048575) gid 0 flags: 0x60000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.938186:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [585728->1048575] (req 585728->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.938210:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:0.0:1769074762.940148:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62640:x1855007616728320:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 2122us (6048us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.940162:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13824 00000100:00100000:3.0:1769074762.943083:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625087488:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074762.943125:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074762.946963:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616728448 00000100:00100000:0.0:1769074762.946975:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616728448 00010000:00010000:0.0:1769074762.947027:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 12 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.947074:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616728704 00000100:00100000:0.0:1769074762.947080:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616728704 00010000:00010000:0.0:1769074762.947100:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3fd expref: 10 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.947131:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616728576 00000100:00100000:0.0:1769074762.947135:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616728576 00010000:00010000:0.0:1769074762.947150:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 13 pid: 36516 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.947171:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616728832 00000100:00100000:0.0:1769074762.947175:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616728832 00010000:00010000:0.0:1769074762.947193:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 22 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074762.947217:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6848 00000100:00100000:0.0:1769074762.947231:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616728448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.947267:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.947301:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 12 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947317:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 12 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947347:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd52e61 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b40b expref: 12 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947365:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.947431:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616728448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 206us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.947454:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6848 00000100:00100000:0.0:1769074762.947488:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6849 00000100:00100000:0.0:1769074762.947495:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616728704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.947504:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.947510:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3fd expref: 10 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.947519:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3fd expref: 10 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.947533:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd52e5a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3fd expref: 10 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:0.0:1769074762.947546:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074762.947551:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.947560:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.947567:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074762.947595:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.947615:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616728704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 121us (714us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.947621:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6849 00000100:00100000:0.0:1769074762.947633:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6850 00000100:00100000:0.0:1769074762.947639:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616728576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.947646:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.947655:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 14 pid: 36516 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947665:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 14 pid: 36516 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947680:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd52d9d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b37f expref: 14 pid: 36516 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947689:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074762.947709:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616728576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 69us (780us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074762.947716:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6850 00000100:00100000:0.0:1769074762.947729:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6851 00000100:00100000:0.0:1769074762.947736:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616728832:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074762.947743:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074762.947752:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 22 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947764:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 22 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074762.947776:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.947779:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074762.947786:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60080000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.948177:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60080000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074762.948219:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074762.948637:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8361 00000100:00100000:3.0:1769074762.948649:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616729088:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:0.0:1769074762.949173:0:33102:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x70000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 13 pid: 9048 timeout: 1850 lvb_type: 0 00000100:00100000:0.0:1769074762.949191:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625087744:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074762.949315:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00010000:00010000:1.0:1769074762.951110:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd52dc7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a9 expref: 22 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:1.0:1769074762.951138:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dfb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625087104:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074762.951187:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### not re-adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 13 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.951197:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 13 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.951215:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:1.0:1769074762.951295:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62641:x1855007616727936:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 27667us (28535us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.951306:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13823 00010000:00010000:1.0:1769074762.951358:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074762.951366:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074762.951389:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074762.952140:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718600/0x7fe2323adfd52e92 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.952161:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.952169:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074762.952173:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074762.952176:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718000, rc 0) 00000100:00100000:1.0:1769074762.952218:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62644:x1855007616728192:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 27802us (29306us total) trans 8589943869 rc 0/0 00000100:00100000:1.0:1769074762.952228:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13268 00000100:00100000:3.0:1769074762.961022:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.961076:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074762.972721:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625087744:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074762.972752:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074762.979266:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616728832:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 31513us (32297us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074762.979295:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6851 00000100:00100000:3.0:1769074762.980071:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.980167:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074762.980292:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616729088:12345-192.168.204.60@tcp:3:cat.0 Request processed in 31634us (33261us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074762.980304:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8361 00002000:00100000:3.0:1769074762.980315:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5444a6d80 x1855007616729088/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.980326:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a6d80 with x1855007616729088 ext(3145728->4194303) 00002000:00010000:3.0:1769074762.980333:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a6d80 00000100:00100000:3.0:1769074762.980359:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616729472 00000100:00100000:3.0:1769074762.980366:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616729472 00002000:00100000:3.0:1769074762.980400:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (667648->671743) req@ffff89b67a271c00 x1855007616729472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/0 e 0 to 0 dl 1769074773 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074762.980408:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271c00 with x1855007616729472 ext(667648->671743) 00010000:00010000:3.0:1769074762.980416:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 11 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074762.980434:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 11 pid: 9048 timeout: 1850 lvb_type: 0 00002000:00010000:3.0:1769074762.980441:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a271c00 00000100:00100000:3.0:1769074762.980459:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 703 00000100:00100000:3.0:1769074762.980467:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616729472:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074762.980571:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616729600 00000100:00100000:1.0:1769074762.980576:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616729600 00000100:00100000:3.0:1769074762.980606:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616729472:12345-192.168.204.60@tcp:3:cat.0 Request processed in 140us (540us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074762.980614:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13269 00000100:00100000:3.0:1769074762.980614:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 703 00002000:00100000:3.0:1769074762.980620:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (667648->671743) req@ffff89b67a271c00 x1855007616729472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/4536 e 0 to 0 dl 1769074773 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1769074762.980621:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62644:x1855007616729600:12345-192.168.204.60@tcp:101:lfs.0 00002000:00100000:3.0:1769074762.980629:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271c00 with x1855007616729472 ext(667648->671743) 00010000:00010000:3.0:1769074762.980634:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 11 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074762.980639:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074762.980643:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 11 pid: 9048 timeout: 1850 lvb_type: 0 00002000:00010000:3.0:1769074762.980651:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a271c00 00010000:00010000:1.0:1769074762.980662:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd52eae lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b44a expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.980765:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.980783:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.980791:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074762.980844:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074762.980856:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 10 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074762.980869:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625087872:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074762.980919:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.017261:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.017919:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616729728 00000100:00100000:3.0:1769074763.017930:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616729728 00010000:00010000:3.0:1769074763.017970:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 11 pid: 6514 timeout: 1850 lvb_type: 0 00000100:00100000:3.0:1769074763.018017:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6852 00000100:00100000:3.0:1769074763.018030:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616729728:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.018045:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.018051:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 11 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074763.018063:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 11 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074763.018085:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b718000/0x7fe2323adfd52e99 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b43c expref: 11 pid: 6514 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074763.018102:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074763.018108:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.018118:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.018127:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.018157:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.026966:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616729344:12345-192.168.204.60@tcp:3:cat.0 Request processed in 64315us (65941us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074763.026984:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8362 00002000:00100000:0.0:1769074763.027002:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b6491eea00 x1855007616729344/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.027013:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491eea00 with x1855007616729344 ext(3145728->4194303) 00002000:00010000:0.0:1769074763.027021:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491eea00 00000100:00100000:3.0:1769074763.027914:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616729728:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9882us (10659us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.027926:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6852 00000100:00100000:3.0:1769074763.028086:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616728960:12345-192.168.204.60@tcp:3:cat.0 Request processed in 80846us (81113us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074763.028096:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8360 00002000:00100000:3.0:1769074763.028105:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5444a6300 x1855007616728960/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.028115:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a6300 with x1855007616728960 ext(2097152->4194303) 00002000:00010000:3.0:1769074763.028121:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a6300 00000100:00100000:3.0:1769074763.028767:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616729216:12345-192.168.204.60@tcp:3:cat.0 Request processed in 65555us (67687us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074763.028777:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8363 00002000:00100000:3.0:1769074763.028783:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b67a271f80 x1855007616729216/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:298/0 lens 488/440 e 0 to 0 dl 1769074773 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.028791:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271f80 with x1855007616729216 ext(2097152->4194303) 00002000:00010000:3.0:1769074763.028796:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a271f80 00000100:00100000:1.0:1769074763.043064:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.043109:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:1.0:1769074763.044211:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074763.044337:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616729856 00000100:00100000:3.0:1769074763.044346:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616729856 00002000:00100000:3.0:1769074763.044407:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (667648->1048575) req@ffff89b54d1dc380 x1855007616729856/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.044418:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dc380 with x1855007616729856 ext(667648->1048575) 00010000:00010000:3.0:1769074763.044427:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 11 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:3.0:1769074763.044440:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 11 pid: 9048 timeout: 1850 lvb_type: 0 00002000:00010000:3.0:1769074763.044447:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d1dc380 00000100:00100000:3.0:1769074763.044487:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 704 00000100:00100000:3.0:1769074763.044496:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616729856:12345-192.168.204.60@tcp:4:cat.0 00000100:00080000:1.0:1769074763.044680:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074763.047187:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.047451:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616730112 00000100:00100000:0.0:1769074763.047464:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616730112 00000100:00100000:0.0:1769074763.047537:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13826 00000100:00100000:0.0:1769074763.047548:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616730112:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074763.047570:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.047597:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd52ec3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b458 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.047678:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625088384:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.047789:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.054449:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625088000:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.054466:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625088128:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.054489:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625088256:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074763.054518:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->1306623] (req 1048576->1306623) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.054536:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546117000, rc 0) 00000100:00100000:3.0:1769074763.054588:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62641:x1855007616729984:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11239us (11476us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.054596:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13825 00000100:00100000:0.0:1769074763.054867:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616730368 00000100:00100000:0.0:1769074763.054873:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616730368 00010000:00010000:0.0:1769074763.054900:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b412 expref: 11 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074763.054934:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616730240 00000100:00100000:0.0:1769074763.054937:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616730240 00010000:00010000:0.0:1769074763.054948:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b0 expref: 24 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074763.054966:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616730496 00000100:00100000:0.0:1769074763.054968:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616730496 00010000:00010000:0.0:1769074763.054978:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b386 expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00000100:00100000:0.0:1769074763.054997:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6853 00000100:00100000:0.0:1769074763.055005:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616730368:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.055017:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.055039:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b412 expref: 11 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055049:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b412 expref: 11 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055078:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ea00/0x7fe2323adfd52e68 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b412 expref: 11 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055092:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.055119:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616730368:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 115us (2452us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.055126:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6853 00000100:00100000:0.0:1769074763.055169:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6854 00000100:00100000:0.0:1769074763.055175:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616730240:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.055181:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.055189:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b0 expref: 24 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055200:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b0 expref: 24 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055212:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd52dce lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b0 expref: 24 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055222:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.055239:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616730240:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 64us (2521us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.055245:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6854 00000100:00100000:0.0:1769074763.055256:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6855 00000100:00100000:0.0:1769074763.055262:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616730496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.055269:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.055277:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b386 expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055523:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b386 expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055536:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd52db2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b386 expref: 15 pid: 27844 timeout: 1850 lvb_type: 1 00010000:00010000:0.0:1769074763.055546:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.055548:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.055555:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.055565:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.055598:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.055613:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625088768:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074763.055659:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.058129:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:0.0:1769074763.058375:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074756 -> 1769074762 00002000:00010000:0.0:1769074763.058381:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074750 -> 1769074762 00002000:00010000:0.0:1769074763.058384:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074756 -> 1769074762 00000100:00100000:0.0:1769074763.058404:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625088384:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.058451:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd52ec3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b458 expref: 12 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.058491:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd52ec3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b458 expref: 12 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.058504:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:0.0:1769074763.058580:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616730112:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11033us (11407us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.058591:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13826 00000100:00100000:0.0:1769074763.058625:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616731008 00000100:00100000:0.0:1769074763.058631:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616731008 00000100:00100000:0.0:1769074763.058681:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13830 00000100:00100000:0.0:1769074763.058691:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616731008:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074763.058707:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.058724:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52edf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b474 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.058775:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625088896:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.059538:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.070312:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.073202:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625088512:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074763.073239:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52eca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b45f expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.073263:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4a00/0x7fe2323adfd52eca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b45f expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.073282:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b4a00, rc 0) 00000100:00100000:3.0:1769074763.073354:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616730624:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20457us (20595us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074763.073364:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13827 00000100:00100000:0.0:1769074763.074099:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625088768:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074763.074120:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.074156:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616730496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 18893us (21417us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.074165:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6855 00002000:00010000:0.0:1769074763.074202:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074761 -> 1769074762 00002000:00010000:0.0:1769074763.074206:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074760 -> 1769074762 00002000:00010000:0.0:1769074763.074208:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074761 -> 1769074762 00000100:00100000:0.0:1769074763.074220:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625088896:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.074239:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52edf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b474 expref: 24 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.074255:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52edf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b474 expref: 24 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.074267:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:0.0:1769074763.074290:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616731008:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15601us (16172us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.074296:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13830 00000100:00100000:1.0:1769074763.079503:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.080936:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616729856:12345-192.168.204.60@tcp:4:cat.0 Request processed in 36439us (37882us total) trans 17179874488 rc 0/0 00000100:00100000:1.0:1769074763.080945:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 704 00002000:00100000:1.0:1769074763.080952:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (667648->1048575) req@ffff89b54d1dc380 x1855007616729856/t17179874488(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/448 e 0 to 0 dl 1769074774 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.080960:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dc380 with x1855007616729856 ext(667648->1048575) 00010000:00010000:1.0:1769074763.080968:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 13 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074763.080984:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 13 pid: 9048 timeout: 1850 lvb_type: 0 00002000:00010000:1.0:1769074763.080992:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d1dc380 00000100:00100000:3.0:1769074763.081669:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.082047:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616731648 00000100:00100000:0.0:1769074763.082055:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616731648 00000100:00100000:0.0:1769074763.082103:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4629 00000100:00100000:0.0:1769074763.082111:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62644:x1855007616731648:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074763.082337:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62644:x1855007616731648:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 226us (672us total) trans 8589943870 rc 0/0 00000100:00100000:0.0:1769074763.082345:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4629 00000100:00100000:3.0:1769074763.094060:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.094101:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:0.0:1769074763.095128:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074763.095200:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616731776 00000100:00100000:1.0:1769074763.095206:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616731776 00002000:00100000:1.0:1769074763.095255:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67a271880 x1855007616731776/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.095267:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271880 with x1855007616731776 ext(4194304->4808703) 00002000:00010000:1.0:1769074763.095275:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a271880 00000100:00100000:1.0:1769074763.095302:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8365 00000100:00100000:1.0:1769074763.095310:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616731776:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074763.109929:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.117698:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.119552:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616732288 00000100:00100000:1.0:1769074763.119569:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616732288 00010000:00010000:1.0:1769074763.119621:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 15 pid: 9048 timeout: 1850 lvb_type: 0 00000100:00100000:1.0:1769074763.119677:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6856 00000100:00100000:1.0:1769074763.119688:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616732288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.119703:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.119726:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 15 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074763.119741:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 15 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074763.119770:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd52e8b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 667648->1048575) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b435 expref: 15 pid: 9048 timeout: 1850 lvb_type: 0 00010000:00010000:1.0:1769074763.119788:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.119790:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.119798:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.119849:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.119888:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.119901:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625089024:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074763.119998:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.133118:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616731776:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37798us (39005us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074763.133132:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8365 00002000:00100000:1.0:1769074763.133143:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67a271880 x1855007616731776/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.133155:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a271880 with x1855007616731776 ext(4194304->4808703) 00002000:00010000:1.0:1769074763.133160:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a271880 00000100:00100000:0.0:1769074763.135674:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616732160:12345-192.168.204.60@tcp:3:cat.0 Request processed in 25433us (25743us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074763.135689:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8366 00000100:00100000:1.0:1769074763.135693:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625089024:192.168.204.60@tcp:105: 00002000:00100000:0.0:1769074763.135703:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5475db480 x1855007616732160/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.135715:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475db480 with x1855007616732160 ext(4194304->4808703) 00010000:00010000:1.0:1769074763.135720:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00010000:0.0:1769074763.135721:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475db480 00000100:00100000:1.0:1769074763.135747:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616732288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16061us (18060us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.135756:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6856 00000100:00100000:0.0:1769074763.147127:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.147753:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.148147:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616732928 00000100:00100000:0.0:1769074763.148152:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616732928 00000100:00100000:0.0:1769074763.148194:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13831 00000100:00100000:0.0:1769074763.148202:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616732928:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074763.148214:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.148233:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52ef4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b482 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.148298:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625089152:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.148371:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.149008:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.160487:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625089152:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.160546:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52ef4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b482 expref: 24 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.160572:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52ef4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b482 expref: 24 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.160587:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:0.0:1769074763.160658:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616732928:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12456us (12905us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.160667:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13831 00010000:00010000:3.0:1769074763.165850:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074763.167180:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.175244:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.175431:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.175537:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616733312 00000100:00100000:0.0:1769074763.175547:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616733312 00000100:00100000:0.0:1769074763.175604:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616733440 00000100:00100000:0.0:1769074763.175608:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616733440 00000100:00100000:0.0:1769074763.175636:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13834 00000100:00100000:0.0:1769074763.175646:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616733312:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074763.175665:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.175683:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52f09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b497 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.175742:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625089536:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.175853:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.176034:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13835 00000100:00100000:1.0:1769074763.176044:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62640:x1855007616733440:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074763.176052:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.176061:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b49e expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.176079:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.176087:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074763.176088:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074763.176092:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->1343487] (req 1048576->1343487) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.176118:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.176130:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 13 pid: 9048 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.176141:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625089664:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074763.176184:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.181929:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.182492:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616733696 00000100:00100000:0.0:1769074763.182502:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616733696 00000100:00100000:0.0:1769074763.182569:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4631 00000100:00100000:0.0:1769074763.182582:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c5450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:62642:x1855007616733696:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:0.0:1769074763.182895:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c5450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62642:x1855007616733696:12345-192.168.204.60@tcp:35:cat.0 Request processed in 314us (976us total) trans 8589943872 rc 0/0 00000100:00100000:0.0:1769074763.182904:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4631 00000100:00100000:1.0:1769074763.183534:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625089536:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074763.183579:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52f09 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b497 expref: 26 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.183606:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52f09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b497 expref: 26 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.183619:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:1.0:1769074763.183662:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616733312:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8017us (8427us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074763.183669:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13834 00000100:00100000:3.0:1769074763.189992:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.208937:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.209048:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616734080 00000100:00100000:1.0:1769074763.209058:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616734080 00000100:00100000:1.0:1769074763.209104:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4632 00000100:00100000:1.0:1769074763.209113:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:62643:x1855007616734080:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074763.209367:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62643:x1855007616734080:12345-192.168.204.60@tcp:35:cat.0 Request processed in 253us (439us total) trans 8589943873 rc 0/0 00000100:00100000:1.0:1769074763.209376:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4632 00000100:00100000:0.0:1769074763.231063:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.231193:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616734208 00000100:00100000:1.0:1769074763.231205:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616734208 00010000:00010000:1.0:1769074763.231249:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 16 pid: 9048 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.231638:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6857 00000100:00100000:1.0:1769074763.231651:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616734208:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.231667:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.231690:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 16 pid: 9048 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.231705:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 16 pid: 9048 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.231730:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd52ebc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1306623) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b451 expref: 16 pid: 9048 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.231746:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.231748:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.231756:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.231766:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.231803:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.231858:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625089792:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074763.231932:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.237288:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625089792:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074763.237322:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.237389:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616734208:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5739us (6336us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.237414:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6857 00000100:00100000:0.0:1769074763.242419:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.242745:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8368 00000100:00100000:2.0:1769074763.242757:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616734336:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074763.242972:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616734336:12345-192.168.204.60@tcp:3:cat.0 Request processed in 216us (575us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074763.242979:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8368 00002000:00100000:2.0:1769074763.242987:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1339392->1343487) req@ffff89b5475daa00 x1855007616734336/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/4536 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.242995:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475daa00 with x1855007616734336 ext(1339392->1343487) 00002000:00010000:2.0:1769074763.242999:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475daa00 00000100:00100000:0.0:1769074763.248358:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.248757:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616734592 00000100:00100000:1.0:1769074763.248768:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616734592 00000100:00100000:1.0:1769074763.248866:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4633 00000100:00100000:1.0:1769074763.248876:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:62640:x1855007616734592:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074763.249141:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62640:x1855007616734592:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 266us (794us total) trans 8589943874 rc 0/0 00000100:00100000:1.0:1769074763.249150:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4633 00000100:00100000:3.0:1769074763.363413:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.364007:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616734848 00000100:00100000:1.0:1769074763.364019:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616734848 00000100:00100000:1.0:1769074763.364090:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13271 00000100:00100000:1.0:1769074763.364101:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62651:x1855007616734848:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074763.364127:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.364150:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd52f17 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4a5 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.364234:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364252:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364277:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364305:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364311:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364317:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364433:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364442:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364448:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074763.364456:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074763.364548:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364555:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364561:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364567:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364573:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074763.364577:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364584:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364597:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074763.364600:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074763.364605:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364611:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd52f1e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364625:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364633:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364638:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364643:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910d000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074763.364645:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364651:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.364658:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074763.365263:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074763.365270:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.365278:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d000/0x7fe2323adfd52f25 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.365441:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd52f17 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.365455:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 12 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.365472:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074763.365475:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074763.365477:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d200, rc 0) 00000100:00100000:1.0:1769074763.365537:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62651:x1855007616734848:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1438us (2134us total) trans 8589943875 rc 0/0 00000100:00100000:1.0:1769074763.365545:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13271 00000100:00100000:0.0:1769074763.410444:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.410565:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616734976 00000100:00100000:2.0:1769074763.410575:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616734976 00000100:00100000:2.0:1769074763.410646:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13836 00000100:00100000:2.0:1769074763.410656:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62651:x1855007616734976:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074763.410675:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.410696:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4ac expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.410736:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 16 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.410746:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.410756:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.410768:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074763.410770:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074763.410774:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1949696->2097151] (req 1949696->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 16 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.410849:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.410863:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 23 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:2.0:1769074763.410877:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625089920:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074763.410963:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.410973:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 9 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:2.0:1769074763.410981:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625090048:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074763.411008:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 16 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.411016:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 16 pid: 36516 timeout: 1851 lvb_type: 1 00000100:00100000:2.0:1769074763.411029:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625090176:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074763.411050:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.416512:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.416589:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.416622:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.416645:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.417212:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616735488 00000100:00100000:1.0:1769074763.417224:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616735488 00010000:00010000:1.0:1769074763.417269:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 10 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.417311:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616735360 00000100:00100000:1.0:1769074763.417315:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616735360 00010000:00010000:1.0:1769074763.417330:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 24 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.417348:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6858 00000100:00100000:1.0:1769074763.417357:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616735488:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.417371:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.417394:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 10 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.417406:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 10 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.417434:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.417495:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616735488:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (906us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.417503:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6858 00000100:00100000:1.0:1769074763.417522:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616735616 00000100:00100000:1.0:1769074763.417525:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616735616 00010000:00010000:1.0:1769074763.417536:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 17 pid: 36516 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.417554:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6859 00000100:00100000:1.0:1769074763.417560:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616735360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.417568:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.417576:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 24 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.417586:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 24 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.417599:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.417618:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616735360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 58us (996us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.417624:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6859 00000100:00100000:1.0:1769074763.417636:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6860 00000100:00100000:1.0:1769074763.417642:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efff450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616735616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.417650:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.417658:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 17 pid: 36516 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.417668:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 17 pid: 36516 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.417680:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.417682:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.417689:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.417700:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.417734:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.417748:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625090304:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074763.417798:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5180 going to sleep for 11 seconds 00010000:00010000:1.0:1769074763.417868:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd52dd5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3b7 expref: 23 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.417888:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625089920:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.417903:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd52e6f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b419 expref: 9 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.417912:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625090048:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.417922:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6c800/0x7fe2323adfd52da4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b38d expref: 17 pid: 36516 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.417930:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625090176:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.417956:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 16 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.417966:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:1.0:1769074763.417988:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62651:x1855007616734976:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 7334us (7559us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.417995:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13836 00000100:00100000:2.0:1769074763.418102:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616735232 00000100:00100000:2.0:1769074763.418108:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616735232 00000100:00100000:2.0:1769074763.418153:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13272 00000100:00100000:2.0:1769074763.418161:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:62653:x1855007616735232:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074763.418181:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.418197:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f41 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4b3 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.418267:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418279:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418289:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418303:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418309:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418314:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418430:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418442:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418449:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074763.418457:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074763.418548:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418555:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418561:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418567:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418572:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.418576:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418585:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418595:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074763.418598:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.418602:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418609:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f48 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418624:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418630:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418635:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418640:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.418643:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418648:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418656:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b643a80100) 00010000:00010000:2.0:1769074763.418658:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.418661:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418666:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f4f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.418700:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f41 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 226 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.418712:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 225 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.418719:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.418722:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074763.418724:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:2.0:1769074763.418758:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:62653:x1855007616735232:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 597us (2256us total) trans 8589943876 rc 0/0 00000100:00100000:2.0:1769074763.418765:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13272 00000100:00100000:0.0:1769074763.430493:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.430837:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625090304:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074763.430862:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.430920:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efff450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616735616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13276us (14273us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.430932:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6860 00000100:00100000:2.0:1769074763.431294:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616735744 00000100:00100000:2.0:1769074763.431303:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616735744 00002000:00100000:2.0:1769074763.431347:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1949696->1953791) req@ffff89b549f28a80 x1855007616735744/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.431356:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28a80 with x1855007616735744 ext(1949696->1953791) 00002000:00010000:2.0:1769074763.431362:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f28a80 00000100:00100000:2.0:1769074763.431389:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8369 00000100:00100000:2.0:1769074763.431398:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616735744:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074763.431552:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616735744:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 154us (1072us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074763.431559:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8369 00002000:00100000:2.0:1769074763.431565:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1949696->1953791) req@ffff89b549f28a80 x1855007616735744/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/4536 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.431573:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28a80 with x1855007616735744 ext(1949696->1953791) 00002000:00010000:2.0:1769074763.431577:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f28a80 00000100:00100000:3.0:1769074763.455124:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.455292:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.456446:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736000 00000100:00100000:2.0:1769074763.456470:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736000 00000100:00100000:2.0:1769074763.456545:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13273 00000100:00100000:2.0:1769074763.456556:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62657:x1855007616736000:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074763.456580:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.456604:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd52f64 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4ba expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.456636:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736128 00000100:00100000:1.0:1769074763.456642:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736128 00010000:00010000:2.0:1769074763.456728:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00000100:00100000:1.0:1769074763.456743:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13837 00010000:00010000:2.0:1769074763.456744:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00000100:00100000:1.0:1769074763.456751:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62651:x1855007616736128:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074763.456755:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.456758:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000004:00200000:2.0:1769074763.456767:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074763.456770:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4c1 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.456792:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 13 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.456801:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.456830:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.456837:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.456839:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.456844:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->1232895] (req 1048576->1232895) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 13 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.456881:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.456894:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 20 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.456908:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090432:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074763.456908:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd52f64 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.456927:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.456936:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.456938:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1769074763.456941:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fa3d600, rc 0) 00010000:00010000:1.0:1769074763.457007:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.457016:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 10 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.457025:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090560:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074763.459902:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62657:x1855007616736000:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3338us (4610us total) trans 8589943877 rc 0/0 00000100:00100000:2.0:1769074763.459914:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13273 00010000:00010000:1.0:1769074763.462039:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 13 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.462059:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 13 pid: 28955 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.462076:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090688:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074763.462164:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074763.465436:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.465842:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.465867:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.465890:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736384 00000100:00100000:0.0:1769074763.465899:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736384 00010000:00010000:0.0:1769074763.465934:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 21 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:0.0:1769074763.465964:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736640 00000100:00100000:0.0:1769074763.465967:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736640 00010000:00010000:0.0:1769074763.465980:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 14 pid: 28955 timeout: 1851 lvb_type: 1 00000100:00100000:0.0:1769074763.465999:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6861 00000100:00100000:0.0:1769074763.466008:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616736384:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.466023:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.466048:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 21 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:0.0:1769074763.466061:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 21 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:0.0:1769074763.466087:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.466137:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616736384:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 129us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.466145:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6861 00000100:00100000:0.0:1769074763.466161:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6862 00000100:00100000:0.0:1769074763.466167:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616736640:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.466175:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.466183:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 14 pid: 28955 timeout: 1851 lvb_type: 1 00010000:00010000:0.0:1769074763.466191:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 14 pid: 28955 timeout: 1851 lvb_type: 1 00010000:00010000:0.0:1769074763.466203:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.466220:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616736640:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 53us (352us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.466226:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6862 00000100:00100000:0.0:1769074763.466305:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.466326:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736512 00000100:00100000:0.0:1769074763.466330:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736512 00010000:00010000:0.0:1769074763.466344:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 11 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:0.0:1769074763.466362:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6863 00000100:00100000:0.0:1769074763.466368:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616736512:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.466375:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.466382:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 11 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:0.0:1769074763.466391:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 11 pid: 27844 timeout: 1851 lvb_type: 1 00010000:00010000:0.0:1769074763.466407:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.466409:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.466415:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 13 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.466425:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 13 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.466461:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 13 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.466848:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e000/0x7fe2323adfd52ddc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3be expref: 20 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.466862:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090432:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.466876:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f200/0x7fe2323adfd52e76 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b420 expref: 11 pid: 27844 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.466885:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090560:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.466893:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd52db9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b394 expref: 13 pid: 28955 timeout: 1851 lvb_type: 1 00000100:00100000:1.0:1769074763.466905:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090688:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.466932:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.466942:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:1.0:1769074763.466971:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62651:x1855007616736128:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 10222us (11862us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.466979:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13837 00000100:00100000:2.0:1769074763.467876:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736256 00000100:00100000:2.0:1769074763.467887:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736256 00000100:00100000:2.0:1769074763.467940:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13274 00000100:00100000:2.0:1769074763.467948:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62656:x1855007616736256:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074763.467968:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.467987:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd52f80 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4c8 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.468050:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468063:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468074:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468090:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468096:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468101:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468169:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468178:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468185:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074763.468228:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:2.0:1769074763.468233:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074763.468254:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074763.468726:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625090816:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074763.468839:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00010000:00010000:2.0:1769074763.468925:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468937:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468942:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468947:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468953:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468959:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.468963:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468970:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468981:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074763.468983:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.468988:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.468993:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd52f87 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469007:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469013:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469018:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469024:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.469026:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469032:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469038:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b643a80100) 00010000:00010000:2.0:1769074763.469040:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.469043:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469048:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd52f8e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.469080:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd52f80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c8 expref: 12 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.469094:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c8 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.469102:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.469105:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074763.469108:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb000, rc 0) 00000100:00100000:2.0:1769074763.469148:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616736256:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1200us (3729us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.469157:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13274 00000100:00100000:2.0:1769074763.477515:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625090816:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074763.477543:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074763.477629:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616736512:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11257us (11321us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.477639:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6863 00000100:00100000:0.0:1769074763.480695:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.480744:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.480787:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.480978:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736768 00000100:00100000:1.0:1769074763.480990:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736768 00000100:00100000:1.0:1769074763.481044:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616736896 00000100:00100000:1.0:1769074763.481047:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616736896 00000100:00100000:1.0:1769074763.481070:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13838 00000100:00100000:1.0:1769074763.481079:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616736768:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074763.481097:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.481118:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd52f9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4cf expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.481176:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090944:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074763.481248:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.481327:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13839 00000100:00100000:2.0:1769074763.481337:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616736896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074763.481345:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.481355:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641650800/0x7fe2323adfd52fa3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4d6 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074763.481385:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625091072:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074763.481410:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.481481:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737024 00000100:00100000:3.0:1769074763.481485:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737024 00000100:00100000:3.0:1769074763.481529:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 989 00000100:00100000:3.0:1769074763.481536:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616737024:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.481546:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.481567:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd52f95 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c8 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.481584:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074763.481590:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.481617:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616737024:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 81us (829us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.481624:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 989 00000100:00100000:3.0:1769074763.489839:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.489871:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.489904:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.489975:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.490171:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737152 00000100:00100000:2.0:1769074763.490183:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737152 00000100:00100000:2.0:1769074763.490254:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737280 00000100:00100000:2.0:1769074763.490259:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737280 00000100:00100000:2.0:1769074763.490278:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737536 00000100:00100000:2.0:1769074763.490282:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737536 00000100:00100000:2.0:1769074763.490296:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737408 00000100:00100000:2.0:1769074763.490299:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737408 00000100:00100000:2.0:1769074763.490319:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13840 00000100:00100000:2.0:1769074763.490328:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616737152:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074763.490348:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.490370:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641651a00/0x7fe2323adfd52faa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4dd expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074763.490436:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625091200:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074763.490532:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.490555:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13841 00000100:00100000:2.0:1769074763.490563:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616737280:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074763.490572:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.490582:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd52fb1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4e4 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074763.490612:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625091328:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074763.490636:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.490774:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13842 00000100:00100000:3.0:1769074763.490784:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616737536:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074763.490793:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074763.490802:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676030800/0x7fe2323adfd52fb8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4f2 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074763.490865:0:8399:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8399:1855007625091456:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074763.490893:0:8399:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.491070:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13843 00000100:00100000:1.0:1769074763.491083:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616737408:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074763.491091:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.491102:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52fbf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4eb expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.491137:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625091584:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074763.491169:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.493903:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625090944:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074763.493953:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd52f9c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4cf expref: 13 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.493987:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd52f9c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4cf expref: 13 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.494001:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:1.0:1769074763.494081:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616736768:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13002us (13395us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074763.494092:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13838 00000100:00100000:2.0:1769074763.495599:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625091072:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074763.495631:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641650800/0x7fe2323adfd52fa3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4d6 expref: 16 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.495649:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641650800/0x7fe2323adfd52fa3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4d6 expref: 16 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.495661:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650800, rc 0) 00000100:00100000:2.0:1769074763.495734:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616736896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 14399us (14989us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074763.495742:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13839 00000100:00100000:3.0:1769074763.499512:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625091200:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074763.499555:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b641651a00/0x7fe2323adfd52faa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4dd expref: 13 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.499576:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641651a00/0x7fe2323adfd52faa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4dd expref: 13 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.499593:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:3.0:1769074763.499662:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616737152:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9333us (9866us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074763.499673:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13840 00002000:00010000:1.0:1769074763.500668:0:8399:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074760 -> 1769074763 00002000:00010000:1.0:1769074763.500676:0:8399:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074758 -> 1769074762 00002000:00010000:1.0:1769074763.500678:0:8399:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074760 -> 1769074763 00000100:00100000:1.0:1769074763.500694:0:8399:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8399:1855007625091456:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074763.500729:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676030800/0x7fe2323adfd52fb8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f2 expref: 11 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.500749:0:8399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676030800/0x7fe2323adfd52fb8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f2 expref: 11 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.500760:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030800, rc 0) 00000100:00100000:1.0:1769074763.500847:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616737536:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 10064us (10941us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074763.500856:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13842 00002000:00010000:3.0:1769074763.502449:0:10284:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074760 -> 1769074763 00002000:00010000:3.0:1769074763.502459:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074761 -> 1769074762 00002000:00010000:3.0:1769074763.502473:0:10284:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074760 -> 1769074763 00000100:00100000:3.0:1769074763.502493:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ec000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625091584:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.502763:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625091328:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.502791:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd52fb1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4e4 expref: 12 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.502830:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd52fb1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4e4 expref: 12 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.502842:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651e00, rc 0) 00000100:00100000:0.0:1769074763.502899:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616737280:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 12336us (13027us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.502909:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13841 00010000:00010000:3.0:1769074763.503002:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52fbf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4eb expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.503015:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52fbf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4eb expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.503023:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:3.0:1769074763.503045:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616737408:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11966us (13070us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074763.503051:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13843 00000100:00100000:2.0:1769074763.507426:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.507784:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737664 00000100:00100000:1.0:1769074763.507800:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737664 00000100:00100000:2.0:1769074763.508281:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.508340:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616737792 00000100:00100000:3.0:1769074763.508346:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616737792 00000100:00100000:3.0:1769074763.508433:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13844 00000100:00100000:3.0:1769074763.508444:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62657:x1855007616737792:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074763.508478:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074763.508499:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b4f9 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.508532:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.508541:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074763.508543:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074763.508547:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 13 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.508583:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.508596:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 19 pid: 28955 timeout: 1851 lvb_type: 0 00000100:00100000:3.0:1769074763.508609:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625091712:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.508692:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00002000:00100000:1.0:1769074763.510076:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1228800->1232895) req@ffff89b5497ab480 x1855007616737664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.510089:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497ab480 with x1855007616737664 ext(1228800->1232895) 00002000:00010000:1.0:1769074763.510096:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497ab480 00000100:00100000:1.0:1769074763.510128:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8370 00000100:00100000:1.0:1769074763.510136:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616737664:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074763.511028:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.511166:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616737664:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 1029us (3753us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074763.511176:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8370 00002000:00100000:1.0:1769074763.511187:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1228800->1232895) req@ffff89b5497ab480 x1855007616737664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/4536 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.511195:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497ab480 with x1855007616737664 ext(1228800->1232895) 00002000:00010000:1.0:1769074763.511199:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497ab480 00000100:00100000:0.0:1769074763.512139:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616738048 00000100:00100000:0.0:1769074763.512148:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616738048 00000100:00100000:0.0:1769074763.512244:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13275 00000100:00100000:0.0:1769074763.512253:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62656:x1855007616738048:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074763.512286:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.512307:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd52fcd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b500 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512407:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.512422:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512430:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512438:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512446:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 6/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.512492:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512503:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 224 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.512522:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625091840:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.512607:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512616:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 13 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.512625:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625091968:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.512655:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.512664:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 11 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.512671:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625092096:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074763.512690:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074763.522492:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fab100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625091840:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074763.522512:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625092096:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.523077:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625091712:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074763.523117:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 13 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.523143:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030400, rc 0) 00000100:00100000:3.0:1769074763.523218:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62657:x1855007616737792:12345-192.168.204.60@tcp:101:cat.0 Request processed in 14775us (14938us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.523229:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13844 00000100:00100000:0.0:1769074763.523344:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625091968:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.523367:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074763.526137:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.526229:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616738304 00000100:00100000:3.0:1769074763.526238:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616738304 00000100:00100000:3.0:1769074763.526300:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4634 00000100:00100000:3.0:1769074763.526309:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62651:x1855007616738304:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074763.526659:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62651:x1855007616738304:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 350us (531us total) trans 8589943878 rc 0/0 00000100:00100000:3.0:1769074763.526667:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4634 00000100:00100000:2.0:1769074763.528537:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.528744:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.528776:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.528838:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.529023:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616738432 00000100:00100000:2.0:1769074763.529032:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616738432 00010000:00010000:2.0:1769074763.529076:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 12 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.529109:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616738944 00000100:00100000:2.0:1769074763.529112:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616738944 00010000:00010000:2.0:1769074763.529123:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 15 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.529139:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616738816 00000100:00100000:2.0:1769074763.529142:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616738816 00010000:00010000:2.0:1769074763.529153:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 225 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.529169:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6864 00000100:00100000:2.0:1769074763.529178:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616738432:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074763.529192:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074763.529199:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 12 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529209:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 12 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529230:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd52eb5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b44a expref: 12 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529247:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.529252:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529274:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529286:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074763.529334:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616738432:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 157us (592us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.529341:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6864 00000100:00100000:2.0:1769074763.529359:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6865 00000100:00100000:2.0:1769074763.529366:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616738944:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074763.529373:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074763.529378:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 15 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529387:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 15 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529399:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd52f2c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4a5 expref: 15 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529408:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.529412:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529417:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529423:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074763.529440:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616738944:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 75us (663us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.529446:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6865 00000100:00100000:2.0:1769074763.529459:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6866 00000100:00100000:2.0:1769074763.529476:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effe850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:2407:x1855007616738816:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074763.529483:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074763.529487:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 225 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529495:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 225 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529506:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd52f56 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4b3 expref: 225 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.529514:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.529517:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529524:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529531:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.529553:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074763.529570:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effe850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2407:x1855007616738816:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 96us (732us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.529577:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6866 00010000:00010000:0.0:1769074763.530203:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.530219:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074763.530241:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074763.530407:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd52fcd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.530424:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.530432:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074763.530435:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074763.530438:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:0.0:1769074763.530498:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616738048:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 18246us (19473us total) trans 8589943879 rc 0/0 00000100:00100000:0.0:1769074763.530506:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13275 00000100:00100000:0.0:1769074763.530529:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616738688 00000100:00100000:0.0:1769074763.530535:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616738688 00000100:00100000:1.0:1769074763.531038:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13276 00000100:00100000:1.0:1769074763.531049:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:62660:x1855007616738688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074763.531070:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.531086:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52fe2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b507 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.531180:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.531193:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.531203:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074763.531212:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074763.531644:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52fe2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 227 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.531662:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 226 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.531669:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074763.531671:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:1.0:1769074763.531674:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d400, rc 0) 00000100:00100000:1.0:1769074763.531790:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:62660:x1855007616738688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 742us (3261us total) trans 8589943880 rc 0/0 00000100:00100000:1.0:1769074763.531799:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13276 00000100:00100000:2.0:1769074763.545027:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.545115:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739072 00000100:00100000:3.0:1769074763.545126:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739072 00002000:00100000:3.0:1769074763.545180:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (585728->1048575) req@ffff89b5497a9f80 x1855007616739072/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.545192:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9f80 with x1855007616739072 ext(585728->1048575) 00010000:00010000:3.0:1769074763.545203:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 20 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.545220:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 20 pid: 28955 timeout: 1851 lvb_type: 0 00002000:00010000:3.0:1769074763.545229:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a9f80 00000100:00100000:3.0:1769074763.545628:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 706 00000100:00100000:3.0:1769074763.545638:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616739072:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074763.562848:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.563044:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739200 00000100:00100000:1.0:1769074763.563057:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739200 00000100:00100000:1.0:1769074763.563128:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13277 00000100:00100000:1.0:1769074763.563138:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:62660:x1855007616739200:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074763.563163:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.563186:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52ff7 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b50e expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.563299:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.563316:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.563325:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.563366:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.563380:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 10 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.563395:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625092224:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074763.563498:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.566218:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.568935:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739328 00000100:00100000:2.0:1769074763.568950:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739328 00000100:00100000:2.0:1769074763.569011:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13278 00000100:00100000:2.0:1769074763.569021:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616739328:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074763.569044:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.569066:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53005 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b515 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.569158:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.569174:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.569184:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:2.0:1769074763.573499:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616739072:12345-192.168.204.60@tcp:4:cat.0 Request processed in 27850us (28471us total) trans 17179874489 rc 0/0 00000100:00100000:2.0:1769074763.573515:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 706 00002000:00100000:2.0:1769074763.573529:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (585728->1048575) req@ffff89b5497a9f80 x1855007616739072/t17179874489(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/448 e 0 to 0 dl 1769074774 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.573540:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9f80 with x1855007616739072 ext(585728->1048575) 00010000:00010000:2.0:1769074763.573550:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.573565:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.573573:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a9f80 00000100:00100000:2.0:1769074763.577081:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625092224:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074763.577111:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074763.584644:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.584706:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.585384:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739456 00000100:00100000:1.0:1769074763.585396:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739456 00010000:00010000:1.0:1769074763.585438:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 13 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.585486:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739584 00000100:00100000:1.0:1769074763.585490:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739584 00010000:00010000:1.0:1769074763.585505:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.585534:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6867 00000100:00100000:1.0:1769074763.585544:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616739456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.585562:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.585569:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 13 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.585582:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 13 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.585606:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd52fd4 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b500 expref: 13 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.585620:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074763.585626:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.585637:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.585644:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074763.585652:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074763.585658:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074763.585664:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00000100:00100000:3.0:1769074763.585686:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6868 00010000:00010000:1.0:1769074763.585691:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.585695:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616739584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.585705:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.585726:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.585735:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.585751:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd52ea0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 585728->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b443 expref: 22 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.585762:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074763.585764:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074763.585771:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.585781:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.585844:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074763.585859:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625092352:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074763.585934:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.586637:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616739456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1091us (2031us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.586647:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6867 00010000:00010000:1.0:1769074763.586681:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074763.586690:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.587955:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074763.587968:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074763.588018:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:0.0:1769074763.588023:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074763.588044:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074763.588080:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd52ff7 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 228 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.588099:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 227 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.588108:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074763.588111:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074763.588114:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:0.0:1769074763.588175:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62660:x1855007616739200:12345-192.168.204.60@tcp:101:cat.0 Request processed in 25039us (25337us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.588184:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13277 00000004:00200000:1.0:1769074763.588864:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074763.588870:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074763.588889:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074763.588915:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53005 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.588932:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.588939:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074763.588941:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074763.588944:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651e00, rc 0) 00000100:00100000:1.0:1769074763.588978:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616739328:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 19959us (22775us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.588985:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13278 00000100:00100000:3.0:1769074763.596360:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625092352:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074763.596389:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.596441:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616739584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10746us (11734us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.596453:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6868 00000100:00100000:3.0:1769074763.598851:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.598906:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739712 00000100:00100000:0.0:1769074763.598916:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739712 00002000:00100000:0.0:1769074763.598967:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5444a4380 x1855007616739712/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.598977:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a4380 with x1855007616739712 ext(0->1048575) 00002000:00010000:0.0:1769074763.598983:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a4380 00000100:00100000:0.0:1769074763.599015:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8371 00000100:00100000:0.0:1769074763.599024:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616739712:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074763.610993:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.611034:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.611062:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.611085:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.611120:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740096 00000100:00100000:1.0:1769074763.611130:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740096 00000100:00100000:1.0:1769074763.611169:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740352 00000100:00100000:1.0:1769074763.611172:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740352 00000100:00100000:1.0:1769074763.611187:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740480 00000100:00100000:1.0:1769074763.611190:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740480 00000100:00100000:1.0:1769074763.611205:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739840 00000100:00100000:1.0:1769074763.611208:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739840 00000100:00100000:1.0:1769074763.611230:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13845 00000100:00100000:1.0:1769074763.611240:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616740096:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074763.611264:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.611285:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53013 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b52a expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.611294:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13846 00000100:00100000:0.0:1769074763.611303:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616740352:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074763.611311:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.611318:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5301a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b538 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.611344:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625092608:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074763.611344:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625092480:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074763.611420:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13847 00000100:00100000:2.0:1769074763.611426:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13848 00000100:00100000:3.0:1769074763.611428:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616740480:12345-192.168.204.60@tcp:101:lfs.0 00000100:00100000:2.0:1769074763.611434:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616739840:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074763.611436:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.611440:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.611448:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b51c expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.611448:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd53021 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b53f expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.611474:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074763.611476:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074763.611485:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 22 pid: 8399 timeout: 0 lvb_type: 1 00000100:00100000:3.0:1769074763.611488:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625092736:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074763.611505:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 22 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.611518:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924a00, rc 0) 00000100:00100000:0.0:1769074763.612698:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.615832:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.616112:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.616208:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740608 00000100:00100000:0.0:1769074763.616217:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740608 00000100:00100000:0.0:1769074763.616268:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13849 00000100:00100000:0.0:1769074763.616277:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616740608:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074763.616295:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.616313:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5302f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b546 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.616369:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625092864:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.616418:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.617645:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.617710:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.617774:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.618269:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616739968 00000100:00100000:0.0:1769074763.618280:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616739968 00000100:00100000:0.0:1769074763.618329:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740224 00000100:00100000:0.0:1769074763.618333:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740224 00000100:00100000:0.0:1769074763.618360:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13850 00000100:00100000:0.0:1769074763.618369:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b35a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616739968:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074763.618389:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.618410:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b523 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.618439:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.618441:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.618453:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.618491:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 13 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.618505:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:1.0:1769074763.619747:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13851 00000100:00100000:1.0:1769074763.619768:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616740224:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074763.619781:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.619796:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd5303d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b531 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.620363:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625092992:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074763.620473:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f56c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.620666:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616739840:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9230us (9576us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.620676:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13848 00000100:00100000:0.0:1769074763.620690:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b35a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616739968:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 193us (854us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.620699:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13850 00002000:00010000:3.0:1769074763.624359:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074762 -> 1769074763 00002000:00010000:3.0:1769074763.624370:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074762 -> 1769074763 00000100:00100000:3.0:1769074763.624400:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625092480:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074763.624446:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53013 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b52a expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.624482:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53013 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b52a expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074763.624495:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074763.624559:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616740096:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 13319us (13574us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074763.624576:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13845 00002000:00010000:0.0:1769074763.624761:0:28954:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074762 -> 1769074763 00000100:00100000:0.0:1769074763.624772:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625092608:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.624794:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5301a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b538 expref: 24 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.624833:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5301a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b538 expref: 24 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.624842:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:0.0:1769074763.624868:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616740352:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13566us (13833us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.624876:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13846 00002000:00010000:1.0:1769074763.630803:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074762 -> 1769074763 00000100:00100000:1.0:1769074763.630875:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625092736:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074763.630911:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd53021 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b53f expref: 12 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.630937:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd53021 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b53f expref: 12 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.630962:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031400, rc 0) 00000100:00100000:1.0:1769074763.631019:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616740480:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 19591us (19956us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074763.631029:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13847 00000100:00100000:2.0:1769074763.631405:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625092992:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074763.631427:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd5303d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b531 expref: 10 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.631439:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd5303d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b531 expref: 10 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074763.631453:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d800, rc 0) 00000100:00100000:2.0:1769074763.631499:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2406:x1855007616740224:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11741us (13724us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074763.631506:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13851 00000100:00100000:0.0:1769074763.631534:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625092864:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.631552:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5302f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b546 expref: 23 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.631574:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5302f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b546 expref: 23 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.631583:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:0.0:1769074763.631604:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616740608:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15328us (15496us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.631610:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13849 00000100:00100000:0.0:1769074763.635231:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616739712:12345-192.168.204.60@tcp:3:cat.0 Request processed in 36195us (36375us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074763.635246:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8371 00002000:00100000:0.0:1769074763.635267:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5444a4380 x1855007616739712/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.635277:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a4380 with x1855007616739712 ext(0->1048575) 00002000:00010000:0.0:1769074763.635283:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a4380 00000100:00100000:3.0:1769074763.640106:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.640233:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740864 00000100:00100000:2.0:1769074763.640245:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740864 00000100:00100000:2.0:1769074763.640302:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13279 00000100:00100000:2.0:1769074763.640313:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616740864:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074763.640332:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.640352:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53044 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b54d expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.640481:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640497:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640508:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640523:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640529:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640536:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640547:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5481b6200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.640551:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640558:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640583:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.640586:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074763.640591:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640597:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd5304b lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.640628:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53044 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b54d expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.640638:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.640640:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074763.640644:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53044 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b54d expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.640652:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924600, rc 0) 00000100:00100000:2.0:1769074763.640698:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616740864:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 386us (602us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074763.640706:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13279 00000100:00100000:1.0:1769074763.642725:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.642915:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616740992 00000100:00100000:2.0:1769074763.642924:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616740992 00000100:00100000:2.0:1769074763.642992:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13852 00000100:00100000:2.0:1769074763.643004:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62653:x1855007616740992:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074763.643025:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.643050:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b554 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.643083:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 14 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.643092:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074763.643095:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074763.643098:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1335296->2011135] (req 1335296->2011135) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.643140:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 14 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.643154:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 14 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.643169:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625093120:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074763.644387:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.645178:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.645256:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616741248 00000100:00100000:3.0:1769074763.645264:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616741248 00000100:00100000:3.0:1769074763.645339:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4635 00000100:00100000:3.0:1769074763.645348:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62656:x1855007616741248:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074763.645649:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62656:x1855007616741248:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 301us (473us total) trans 8589943881 rc 0/0 00000100:00100000:3.0:1769074763.645656:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4635 00000100:00100000:0.0:1769074763.646468:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.646543:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616741376 00000100:00100000:1.0:1769074763.646549:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616741376 00000100:00100000:2.0:1769074763.646660:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625093120:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074763.646689:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1335296->2011135] (req 1335296->2011135) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.646707:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b7e00, rc 0) 00002000:00100000:1.0:1769074763.646795:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1949696->2097151) req@ffff89b547ef7480 x1855007616741376/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.647071:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7480 with x1855007616741376 ext(1949696->2097151) 00010000:00010000:1.0:1769074763.647080:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 15 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.647094:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 15 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:1.0:1769074763.647102:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef7480 00000100:00100000:1.0:1769074763.647128:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 707 00000100:00100000:1.0:1769074763.647138:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616741376:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074763.647291:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62653:x1855007616740992:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 4288us (4574us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.647299:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13852 00000100:00100000:3.0:1769074763.658537:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.659154:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616741504 00000100:00100000:1.0:1769074763.659164:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616741504 00002000:00100000:1.0:1769074763.659206:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5444a7b80 x1855007616741504/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.659217:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a7b80 with x1855007616741504 ext(0->1048575) 00002000:00010000:1.0:1769074763.659223:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a7b80 00000100:00100000:1.0:1769074763.659257:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8372 00000100:00100000:1.0:1769074763.659266:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616741504:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074763.663975:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616741376:12345-192.168.204.60@tcp:4:cat.0 Request processed in 16827us (17507us total) trans 21474841170 rc 0/0 00000100:00100000:2.0:1769074763.663991:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 707 00002000:00100000:2.0:1769074763.664005:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1949696->2097151) req@ffff89b547ef7480 x1855007616741376/t21474841170(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/448 e 0 to 0 dl 1769074774 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.664016:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7480 with x1855007616741376 ext(1949696->2097151) 00010000:00010000:2.0:1769074763.664026:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 17 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.664054:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 17 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.664062:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef7480 00000100:00100000:1.0:1769074763.677208:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074763.677231:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625093248:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074763.677304:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074763.677356:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074763.677364:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625093376:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074763.677389:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074763.677421:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625093248 00000100:00100000:1.0:1769074763.677429:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625093248 00000100:00100000:1.0:1769074763.677494:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625093376 00000100:00100000:1.0:1769074763.677498:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625093376 00000100:00100000:1.0:1769074763.677519:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2643 00000100:00100000:1.0:1769074763.677526:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625093248:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074763.677560:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6438912 pending 0 free 1446150144 avail 1299910656 00000100:00100000:1.0:1769074763.677603:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625093248:12345-0@lo:41:osp-pre-1-0.0 Request processed in 77us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.677610:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2643 00000100:00100000:1.0:1769074763.677628:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2644 00000100:00100000:1.0:1769074763.677633:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625093376:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074763.677648:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6389760 pending 0 free 1445400576 avail 1299161088 00000100:00100000:1.0:1769074763.677680:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625093376:12345-0@lo:41:osp-pre-0-1.0 Request processed in 46us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.677685:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2644 00000100:00100000:1.0:1769074763.677730:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625093376:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074763.677757:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625093248:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074763.681609:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625093504:0@lo:400:kworker.0 00000100:00100000:1.0:1769074763.681679:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074763.681778:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625093504 00000100:00100000:0.0:1769074763.681787:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625093504 00000100:00100000:0.0:1769074763.681880:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:0.0:1769074763.681888:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b09c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625093504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.682284:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625093888:0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.682318:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074763.682554:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b09c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625093504:12345-0@lo:400:kworker.0 Request processed in 665us (873us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.682564:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:2.0:1769074763.682695:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625093760:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.682723:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074763.682792:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625094272:0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.682959:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625094400:0@lo:400:kworker.0 00000100:00100000:0.0:1769074763.683134:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625093888 00000100:00100000:0.0:1769074763.683141:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625093888 00000100:00100000:0.0:1769074763.683183:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625093760 00000100:00100000:0.0:1769074763.683186:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625093760 00000100:00100000:0.0:1769074763.683210:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1434 00000100:00100000:0.0:1769074763.683217:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625093888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.683294:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074763.683321:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1769074763.683328:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625094016:0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.683350:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074763.683369:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625094528:0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.683389:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074763.683418:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094528 00000100:00100000:3.0:1769074763.683421:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094528 00000100:00100000:3.0:1769074763.683475:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1435 00000100:00100000:3.0:1769074763.683482:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625093760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.683512:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625093760:12345-0@lo:400:kworker.0 Request processed in 31us (788us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.683518:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1435 00000100:00100000:3.0:1769074763.683533:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1436 00000100:00100000:3.0:1769074763.683539:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625094528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.683566:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625094528:12345-0@lo:400:kworker.0 Request processed in 27us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.683572:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1436 00000100:00100000:3.0:1769074763.683617:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625094528:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.684637:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074763.684684:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625094784:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.684719:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074763.684763:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625093760:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.684775:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1769074763.684781:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625093632:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.684831:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074763.685056:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497a9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625094144:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685078:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074763.685089:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625094656:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685116:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074763.685152:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094400 00000100:00100000:2.0:1769074763.685158:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094400 00000100:00100000:2.0:1769074763.685191:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094016 00000100:00100000:2.0:1769074763.685194:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094016 00000100:00100000:2.0:1769074763.685208:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094272 00000100:00100000:2.0:1769074763.685211:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094272 00000100:00100000:2.0:1769074763.685221:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094784 00000100:00100000:2.0:1769074763.685223:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094784 00000100:00100000:2.0:1769074763.685232:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625093632 00000100:00100000:2.0:1769074763.685234:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625093632 00000100:00100000:2.0:1769074763.685243:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094144 00000100:00100000:2.0:1769074763.685245:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094144 00000100:00100000:2.0:1769074763.685270:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2292 00000100:00100000:2.0:1769074763.685278:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625094400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685326:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625094400:12345-0@lo:400:kworker.0 Request processed in 49us (2031us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685333:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2292 00000100:00100000:2.0:1769074763.685349:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2293 00000100:00100000:2.0:1769074763.685355:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625094016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685381:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625094016:12345-0@lo:400:kworker.0 Request processed in 27us (2031us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685387:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2293 00000100:00100000:2.0:1769074763.685397:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2294 00000100:00100000:2.0:1769074763.685403:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625094272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685433:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625094272:12345-0@lo:400:kworker.0 Request processed in 30us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685438:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2294 00000100:00100000:2.0:1769074763.685450:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2295 00000100:00100000:2.0:1769074763.685455:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625094784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685509:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625094784:12345-0@lo:400:kworker.0 Request processed in 54us (789us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685515:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2295 00000100:00100000:2.0:1769074763.685526:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2296 00000100:00100000:2.0:1769074763.685531:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625093632:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685554:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625093632:12345-0@lo:400:kworker.0 Request processed in 23us (749us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685559:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2296 00000100:00100000:2.0:1769074763.685568:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2297 00000100:00100000:2.0:1769074763.685573:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625094144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685596:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625094144:12345-0@lo:400:kworker.0 Request processed in 22us (516us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685601:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2297 00000100:00100000:2.0:1769074763.685622:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625094656 00000100:00100000:2.0:1769074763.685626:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625094656 00000100:00100000:2.0:1769074763.685649:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1437 00000100:00100000:2.0:1769074763.685656:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+23:3652:x1855007625094656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685683:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+23:3652:x1855007625094656:12345-0@lo:400:kworker.0 Request processed in 28us (566us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.685692:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1437 00000100:00100000:2.0:1769074763.685725:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625094016:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685737:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625094400:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685759:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625093632:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685768:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625094144:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685778:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625094656:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685787:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625094272:0@lo:400:kworker.0 00000100:00100000:2.0:1769074763.685797:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625094784:0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.688800:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.689406:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.689656:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616741632 00000100:00100000:2.0:1769074763.689668:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616741632 00002000:00100000:2.0:1769074763.689716:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1343488->2097151) req@ffff89b546f96d80 x1855007616741632/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.689728:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96d80 with x1855007616741632 ext(1343488->2097151) 00002000:00010000:2.0:1769074763.689734:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f96d80 00000100:00100000:2.0:1769074763.689765:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8373 00000100:00100000:2.0:1769074763.689774:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616741632:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074763.691237:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625093504:0@lo:400:kworker.0 00000100:00100000:1.0:1769074763.691300:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616741760 00000100:00100000:1.0:1769074763.691306:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616741760 00000100:00100000:1.0:1769074763.691348:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13853 00000100:00100000:1.0:1769074763.691356:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62657:x1855007616741760:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074763.691370:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.691390:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b55b expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.691421:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.691430:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074763.691432:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074763.691437:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.691478:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.691494:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 24 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.691507:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625094912:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074763.691604:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.697500:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.699526:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616741888 00000100:00100000:2.0:1769074763.699541:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616741888 00010000:00010000:2.0:1769074763.699599:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 19 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.699643:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6869 00000100:00100000:2.0:1769074763.699653:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616741888:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074763.699669:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074763.699692:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 19 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.699705:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 19 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.699728:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd52f3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1949696->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ac expref: 19 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.699745:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074763.699747:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074763.699754:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.699764:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.699802:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074763.700165:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625095040:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074763.700764:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00080000:2.0:1769074763.700866:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074763.700873:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074763.700876:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074763.700879:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074763.707981:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.708013:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742016 00000100:00100000:2.0:1769074763.708022:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742016 00002000:00100000:2.0:1769074763.708074:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1343487) req@ffff89b54441f800 x1855007616742016/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.708085:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f800 with x1855007616742016 ext(1048576->1343487) 00010000:00010000:2.0:1769074763.708093:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.708108:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.708116:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441f800 00000100:00100000:2.0:1769074763.708140:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 708 00000100:00100000:2.0:1769074763.708149:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616742016:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074763.709099:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625093888:12345-0@lo:400:kworker.0 Request processed in 25877us (26775us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.709111:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1434 00000100:00100000:1.0:1769074763.709435:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625094912:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.709475:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.709492:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:1.0:1769074763.709540:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62657:x1855007616741760:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18185us (20137us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.709548:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13853 00000100:00100000:1.0:1769074763.709644:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616741504:12345-192.168.204.60@tcp:3:cat.0 Request processed in 50379us (51116us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074763.709652:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8372 00002000:00100000:1.0:1769074763.709661:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5444a7b80 x1855007616741504/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.709670:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a7b80 with x1855007616741504 ext(0->1048575) 00002000:00010000:1.0:1769074763.709675:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5444a7b80 00000100:00100000:3.0:1769074763.709908:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625093888:0@lo:400:kworker.0 00000100:00100000:3.0:1769074763.712693:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625095040:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074763.712719:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.712794:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616741888:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13141us (15302us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.712833:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6869 00000100:00100000:0.0:1769074763.713179:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.719154:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742144 00000100:00100000:3.0:1769074763.719170:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742144 00002000:00100000:3.0:1769074763.719220:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1335296->1339391) req@ffff89b547ef5500 x1855007616742144/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.719233:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5500 with x1855007616742144 ext(1335296->1339391) 00002000:00010000:3.0:1769074763.719238:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef5500 00000100:00100000:3.0:1769074763.719266:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8374 00000100:00100000:3.0:1769074763.719276:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616742144:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074763.719434:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616742144:12345-192.168.204.60@tcp:3:cat.0 Request processed in 159us (6255us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074763.719446:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8374 00002000:00100000:3.0:1769074763.719452:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1335296->1339391) req@ffff89b547ef5500 x1855007616742144/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/4536 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.719474:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5500 with x1855007616742144 ext(1335296->1339391) 00002000:00010000:3.0:1769074763.719477:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef5500 00000100:00100000:2.0:1769074763.721094:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616741632:12345-192.168.204.60@tcp:3:cat.0 Request processed in 31310us (32293us total) trans 0 rc 753664/753664 00000100:00100000:2.0:1769074763.721108:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8373 00002000:00100000:2.0:1769074763.721119:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1343488->2097151) req@ffff89b546f96d80 x1855007616741632/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 753664/753664 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.721130:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96d80 with x1855007616741632 ext(1343488->2097151) 00010000:00010000:2.0:1769074763.721141:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.721157:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.721167:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f96d80 00000100:00100000:2.0:1769074763.729532:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616742016:12345-192.168.204.60@tcp:4:cat.0 Request processed in 21371us (21548us total) trans 21474841171 rc 0/0 00000100:00100000:2.0:1769074763.729545:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 708 00002000:00100000:2.0:1769074763.729558:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1343487) req@ffff89b54441f800 x1855007616742016/t21474841171(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/448 e 0 to 0 dl 1769074774 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.730025:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f800 with x1855007616742016 ext(1048576->1343487) 00010000:00010000:2.0:1769074763.730036:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.730051:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 26 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.730058:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441f800 00000100:00100000:1.0:1769074763.750680:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.750731:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.750843:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742400 00000100:00100000:3.0:1769074763.750854:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742400 00000100:00100000:2.0:1769074763.750882:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742272 00000100:00100000:2.0:1769074763.750889:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742272 00010000:00010000:3.0:1769074763.750896:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 27 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00100000:2.0:1769074763.750928:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e5500 x1855007616742272/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 504/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.750940:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5500 with x1855007616742272 ext(1048576->2097151) 00000100:00100000:3.0:1769074763.750942:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6870 00000100:00100000:3.0:1769074763.750954:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616742400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00002000:00010000:2.0:1769074763.750960:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e5500 00010000:00010000:3.0:1769074763.750968:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.750995:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 27 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.750997:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8375 00000100:00100000:2.0:1769074763.751011:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616742272:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:3.0:1769074763.751016:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 27 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.751041:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd52f10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1343487) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b49e expref: 27 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.751057:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074763.751060:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074763.751067:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.751082:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.751120:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074763.751135:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625095168:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074763.751218:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.757456:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625095168:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074763.757494:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.757542:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616742400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6588us (6809us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.757553:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6870 00000100:00100000:2.0:1769074763.766429:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.767133:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742528 00000100:00100000:1.0:1769074763.767147:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742528 00000100:00100000:1.0:1769074763.767222:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13854 00000100:00100000:1.0:1769074763.767231:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62660:x1855007616742528:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074763.767253:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.767276:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b562 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.767309:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.767311:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.767321:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.767337:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.767350:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4e00, rc 0) 00000100:00100000:1.0:1769074763.767480:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62660:x1855007616742528:12345-192.168.204.60@tcp:101:cat.0 Request processed in 249us (1070us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.767487:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13854 00000100:00100000:2.0:1769074763.775244:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.776092:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742656 00000100:00100000:0.0:1769074763.776105:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742656 00002000:00100000:0.0:1769074763.776155:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441fb80 x1855007616742656/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.776168:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441fb80 with x1855007616742656 ext(1048576->2097151) 00002000:00010000:0.0:1769074763.776184:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441fb80 00000100:00100000:0.0:1769074763.776643:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8376 00000100:00100000:0.0:1769074763.776659:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616742656:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074763.786127:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616742272:12345-192.168.204.60@tcp:3:cat.0 Request processed in 35104us (35443us total) trans 0 rc 1044480/1044480 00000100:00100000:0.0:1769074763.786143:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8375 00002000:00100000:0.0:1769074763.786157:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e5500 x1855007616742272/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 504/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1044480/1044480 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.786167:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5500 with x1855007616742272 ext(1048576->2097151) 00002000:00010000:0.0:1769074763.786188:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e5500 00000100:00100000:2.0:1769074763.788249:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.789164:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616742784 00000100:00100000:0.0:1769074763.789189:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616742784 00002000:00100000:0.0:1769074763.789254:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441ca80 x1855007616742784/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.789267:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ca80 with x1855007616742784 ext(1048576->2097151) 00002000:00010000:0.0:1769074763.789416:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441ca80 00000100:00100000:0.0:1769074763.789452:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8377 00000100:00100000:0.0:1769074763.789472:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616742784:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074763.816231:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616742656:12345-192.168.204.60@tcp:3:cat.0 Request processed in 39561us (40980us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074763.816248:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8376 00002000:00100000:0.0:1769074763.816264:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441fb80 x1855007616742656/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.816284:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441fb80 with x1855007616742656 ext(1048576->2097151) 00002000:00010000:0.0:1769074763.816292:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441fb80 00000100:00100000:2.0:1769074763.817544:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.818041:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743040 00000100:00100000:3.0:1769074763.818055:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743040 00000100:00100000:3.0:1769074763.818117:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4636 00000100:00100000:3.0:1769074763.818127:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62653:x1855007616743040:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074763.818398:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:62653:x1855007616743040:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 271us (861us total) trans 8589943882 rc 0/0 00000100:00100000:3.0:1769074763.818406:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4636 00000100:00100000:0.0:1769074763.821947:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616742784:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32463us (33696us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074763.821963:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8377 00002000:00100000:0.0:1769074763.821976:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441ca80 x1855007616742784/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.821985:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ca80 with x1855007616742784 ext(1048576->2097151) 00002000:00010000:0.0:1769074763.821991:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441ca80 00000100:00100000:1.0:1769074763.860123:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.860228:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.860556:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743168 00000100:00100000:3.0:1769074763.860567:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743168 00000100:00100000:3.0:1769074763.860631:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13855 00000100:00100000:3.0:1769074763.860642:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62657:x1855007616743168:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074763.860661:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074763.860680:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b569 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.860710:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.860719:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074763.860721:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074763.860725:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.860763:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.860776:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 25 pid: 28955 timeout: 1851 lvb_type: 0 00000100:00100000:3.0:1769074763.860790:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625095296:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.860909:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.862784:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743424 00000100:00100000:2.0:1769074763.862797:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743424 00000100:00100000:2.0:1769074763.862907:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13280 00000100:00100000:2.0:1769074763.862917:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e723850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62664:x1855007616743424:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074763.862937:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.862958:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd5306e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b570 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.863034:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863048:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863059:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863073:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863079:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863085:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863197:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863206:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863213:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074763.863223:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074763.863307:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863314:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863319:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863325:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863331:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.863335:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863341:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863353:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074763.863356:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.863361:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863366:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd53075 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863379:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863389:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863394:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863399:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.863402:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863407:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863414:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4400) 00010000:00010000:2.0:1769074763.863416:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.863419:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863424:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5307c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074763.863475:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd5306e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.863488:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.863494:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.863496:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074763.863499:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867c00, rc 0) 00000100:00100000:2.0:1769074763.866595:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e723850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62664:x1855007616743424:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 3665us (6351us total) trans 8589943883 rc 0/0 00000100:00100000:2.0:1769074763.866609:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13280 00000100:00100000:0.0:1769074763.883855:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.884185:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743680 00000100:00100000:1.0:1769074763.884195:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743680 00002000:00100000:1.0:1769074763.884258:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1335296->2011135) req@ffff89b547ef4000 x1855007616743680/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.884269:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef4000 with x1855007616743680 ext(1335296->2011135) 00010000:00010000:1.0:1769074763.884279:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 26 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.884295:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 26 pid: 28955 timeout: 1851 lvb_type: 0 00002000:00010000:1.0:1769074763.884303:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef4000 00000100:00100000:1.0:1769074763.884340:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 709 00000100:00100000:1.0:1769074763.884349:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616743680:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074763.885275:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625095296:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074763.885405:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.885423:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb000, rc 0) 00000100:00100000:3.0:1769074763.885649:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62657:x1855007616743168:12345-192.168.204.60@tcp:101:cat.0 Request processed in 25007us (25534us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.885660:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13855 00000100:00100000:2.0:1769074763.885875:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.886032:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743552 00000100:00100000:3.0:1769074763.886038:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743552 00000100:00100000:3.0:1769074763.886114:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13856 00000100:00100000:3.0:1769074763.886123:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62660:x1855007616743552:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074763.886136:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074763.886156:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b577 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.886179:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.886187:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074763.886189:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074763.886192:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.886223:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.886234:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 12 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:3.0:1769074763.886247:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625095424:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.886502:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.910663:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616743680:12345-192.168.204.60@tcp:4:cat.0 Request processed in 26300us (26804us total) trans 21474841172 rc 0/0 00000100:00100000:2.0:1769074763.910679:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 709 00002000:00100000:2.0:1769074763.910694:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1335296->2011135) req@ffff89b547ef4000 x1855007616743680/t21474841172(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/448 e 0 to 0 dl 1769074774 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.910705:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef4000 with x1855007616743680 ext(1335296->2011135) 00010000:00010000:2.0:1769074763.910716:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 28 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.910733:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 28 pid: 28955 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.910742:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef4000 00000100:00100000:0.0:1769074763.912448:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625095424:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.912496:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.912516:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba400, rc 0) 00000100:00100000:0.0:1769074763.912567:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62660:x1855007616743552:12345-192.168.204.60@tcp:101:cat.0 Request processed in 26445us (26692us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.912577:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13856 00000100:00100000:1.0:1769074763.912774:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.912972:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743808 00000100:00100000:2.0:1769074763.912980:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743808 00002000:00100000:2.0:1769074763.913026:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1232895) req@ffff89b546f90000 x1855007616743808/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.913035:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90000 with x1855007616743808 ext(1048576->1232895) 00010000:00010000:2.0:1769074763.913042:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 13 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:2.0:1769074763.913055:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 13 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:2.0:1769074763.913066:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f90000 00000100:00100000:2.0:1769074763.913086:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 710 00000100:00100000:2.0:1769074763.913094:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616743808:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074763.913835:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.914137:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616743936 00000100:00100000:2.0:1769074763.914146:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616743936 00000100:00100000:2.0:1769074763.914205:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13281 00000100:00100000:2.0:1769074763.914215:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62666:x1855007616743936:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074763.914235:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074763.914272:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53098 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b57e expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.914336:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914351:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914363:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914377:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914385:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914392:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914476:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914486:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914493:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074763.914534:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:2.0:1769074763.914538:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074763.914555:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914566:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914574:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914579:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914585:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914590:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914595:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.914598:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914604:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914627:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074763.914632:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.914637:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914643:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd5309f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914656:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914661:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914666:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914670:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074763.914673:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914678:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914684:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4a00) 00010000:00010000:2.0:1769074763.914686:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074763.914688:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914692:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd530a6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.914721:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53098 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b57e expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.914735:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b57e expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.914742:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.914744:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074763.914747:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867a00, rc 0) 00000100:00100000:2.0:1769074763.918673:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62666:x1855007616743936:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 4446us (4859us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.918690:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13281 00000100:00100000:3.0:1769074763.930186:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616743808:12345-192.168.204.60@tcp:4:cat.0 Request processed in 17079us (17398us total) trans 17179874490 rc 0/0 00000100:00100000:3.0:1769074763.930201:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 710 00002000:00100000:3.0:1769074763.930216:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1232895) req@ffff89b546f90000 x1855007616743808/t17179874490(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/448 e 0 to 0 dl 1769074774 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074763.930227:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90000 with x1855007616743808 ext(1048576->1232895) 00010000:00010000:3.0:1769074763.930237:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 15 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.930249:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 15 pid: 8398 timeout: 1851 lvb_type: 0 00002000:00010000:3.0:1769074763.930256:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f90000 00000100:00100000:0.0:1769074763.940511:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.940550:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.940580:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.940622:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.940640:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.940693:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.940728:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744320 00000100:00100000:0.0:1769074763.940738:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744320 00000100:00100000:0.0:1769074763.940783:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744448 00000100:00100000:0.0:1769074763.940786:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744448 00000100:00100000:0.0:1769074763.940844:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744576 00000100:00100000:0.0:1769074763.940847:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744576 00000100:00100000:0.0:1769074763.940866:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744704 00000100:00100000:0.0:1769074763.940869:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744704 00000100:00100000:0.0:1769074763.940891:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13857 00000100:00100000:0.0:1769074763.940901:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616744320:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074763.940920:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.940941:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b585 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.940967:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.940969:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.940980:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.940996:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.941015:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:0.0:1769074763.941066:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616744320:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 166us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.941073:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13857 00000100:00100000:0.0:1769074763.941091:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13858 00000100:00100000:0.0:1769074763.941098:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616744448:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074763.941104:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.941112:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd530bb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b58c expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.941157:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625095552:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.941209:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074763.941231:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13859 00000100:00100000:0.0:1769074763.941238:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616744576:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074763.941246:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.941256:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd530c2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b593 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.941284:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625095680:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074763.941308:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074763.941325:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13860 00000100:00100000:0.0:1769074763.941332:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62664:x1855007616744704:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074763.941339:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074763.941347:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b59a expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.941364:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.941373:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 12 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.941381:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 22 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.941387:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.941389:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.941392:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [548864->917503] (req 548864->917503) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.941410:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 22 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.941421:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 22 pid: 8399 timeout: 1851 lvb_type: 1 00000100:00100000:0.0:1769074763.941430:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625095808:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.941471:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 12 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.941480:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 12 pid: 36516 timeout: 1851 lvb_type: 1 00000100:00100000:0.0:1769074763.941489:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625095936:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.941511:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.941519:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 16 pid: 10284 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.941527:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625096064:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074763.941540:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074763.941716:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744064 00000100:00100000:1.0:1769074763.941722:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744064 00010000:00010000:1.0:1769074763.941746:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 28 pid: 28955 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.941772:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744192 00000100:00100000:1.0:1769074763.941775:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744192 00000100:00100000:1.0:1769074763.941794:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6871 00000100:00100000:1.0:1769074763.941800:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616744064:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.941837:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.941857:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 28 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.941870:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 28 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.941887:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.941889:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.941895:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.941904:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.941926:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.941935:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625096192:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074763.941965:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.942067:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 990 00000100:00100000:2.0:1769074763.942076:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616744192:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074763.942086:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074763.942101:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd530ad lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b57e expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.942117:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.942122:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074763.942146:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616744192:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 72us (1597us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074763.942153:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 990 00000100:00100000:1.0:1769074763.945199:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.945438:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744832 00000100:00100000:0.0:1769074763.945464:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744832 00010000:00010000:0.0:1769074763.945513:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 17 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.945559:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6872 00000100:00100000:0.0:1769074763.945569:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effc050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616744832:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.945586:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.945607:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 17 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:0.0:1769074763.945632:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 17 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:0.0:1769074763.945663:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.945666:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074763.945674:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.945685:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074763.945726:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074763.945739:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625096320:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074763.945877:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64a8b3cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074763.954075:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.954097:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.954123:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745088 00000100:00100000:2.0:1769074763.954132:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745088 00010000:00010000:2.0:1769074763.954168:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 23 pid: 8399 timeout: 1851 lvb_type: 1 00000100:00100000:3.0:1769074763.955005:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625095808:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.955020:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625095936:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074763.955030:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625096064:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074763.955066:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [548864->917503] (req 548864->917503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074763.955089:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:3.0:1769074763.955151:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62664:x1855007616744704:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 13818us (14457us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.955161:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13860 00000100:00100000:3.0:1769074763.955216:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616744960 00000100:00100000:3.0:1769074763.955222:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616744960 00010000:00010000:3.0:1769074763.955246:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 13 pid: 36516 timeout: 1851 lvb_type: 1 00000100:00100000:3.0:1769074763.955288:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6873 00000100:00100000:3.0:1769074763.955297:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616744960:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.955311:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.955332:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 13 pid: 36516 timeout: 1851 lvb_type: 1 00010000:00010000:3.0:1769074763.955342:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 13 pid: 36516 timeout: 1851 lvb_type: 1 00010000:00010000:3.0:1769074763.955365:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53036 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b523 expref: 13 pid: 36516 timeout: 1851 lvb_type: 1 00010000:00010000:3.0:1769074763.955377:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.955396:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616744960:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 100us (1298us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.955401:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6873 00000100:00100000:3.0:1769074763.955486:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.955507:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745216 00000100:00100000:3.0:1769074763.955511:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745216 00010000:00010000:3.0:1769074763.955523:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 17 pid: 10284 timeout: 1851 lvb_type: 0 00000100:00100000:3.0:1769074763.955542:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6874 00000100:00100000:3.0:1769074763.955548:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616745216:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.955556:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.955564:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 17 pid: 10284 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.955572:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 17 pid: 10284 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.955585:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd52fc6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4f9 expref: 17 pid: 10284 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.955595:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.956895:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625096192:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074763.956916:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.956956:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616744064:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 15155us (16457us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.956964:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6871 00000100:00100000:1.0:1769074763.956991:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6875 00000100:00100000:1.0:1769074763.956999:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616745088:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074763.957011:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074763.957034:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 23 pid: 8399 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.957049:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 23 pid: 8399 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.957080:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53028 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b51c expref: 23 pid: 8399 timeout: 1851 lvb_type: 1 00010000:00010000:1.0:1769074763.957092:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.957094:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074763.957103:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.957114:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.957154:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074763.957184:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625096448:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074763.957250:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00010000:00010000:1.0:1769074763.957281:0:27844:0:(ldlm_lockd.c:740:ldlm_handle_ast_error()) ### client (nid 192.168.204.60@tcp) returned -22 from glimpse AST (req@0000000035d52c28 x1855007625095680) - normal race ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 16 pid: 8398 timeout: 1851 lvb_type: 0 00010000:00010000:1.0:1769074763.957302:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910de00/0x7fe2323adfd52f72 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1232895) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4c1 expref: 16 pid: 8398 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.957311:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625095680:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074763.957336:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd530c2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b593 expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.957353:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd530c2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b593 expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074763.957361:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:1.0:1769074763.957386:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616744576:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 16148us (16744us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074763.957393:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13859 00000100:00100000:0.0:1769074763.957984:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625096320:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074763.957998:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.958025:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effc050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616744832:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12458us (12857us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.958032:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6872 00010000:00010000:0.0:1769074763.958098:0:9048:0:(ldlm_lockd.c:740:ldlm_handle_ast_error()) ### client (nid 192.168.204.60@tcp) returned -22 from glimpse AST (req@00000000f51b1a2f x1855007625095552) - normal race ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 27 pid: 28955 timeout: 1851 lvb_type: 0 00010000:00010000:0.0:1769074763.958123:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7e00/0x7fe2323adfd53052 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1335296->2011135) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b554 expref: 27 pid: 28955 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.958133:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625095552:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074763.958151:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd530bb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b58c expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.958169:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd530bb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b58c expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074763.958177:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:0.0:1769074763.958199:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616744448:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 17101us (17576us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074763.958205:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13858 00000100:00100000:3.0:1769074763.958973:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616745216:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3419us (3481us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.958983:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6874 00000100:00100000:3.0:1769074763.959097:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.961229:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745344 00000100:00100000:1.0:1769074763.961243:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745344 00002000:00100000:1.0:1769074763.961309:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546c6a300 x1855007616745344/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.961320:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6a300 with x1855007616745344 ext(1048576->2097151) 00002000:00010000:1.0:1769074763.961326:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c6a300 00000100:00100000:1.0:1769074763.961510:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8378 00000100:00100000:1.0:1769074763.961521:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616745344:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074763.970442:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625096448:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074763.970485:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074763.970557:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616745088:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13557us (16491us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074763.970569:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6875 00000100:00100000:3.0:1769074763.971865:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.971903:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074763.975765:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745728 00000100:00100000:1.0:1769074763.975780:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745728 00000100:00100000:1.0:1769074763.975875:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13282 00000100:00100000:1.0:1769074763.975887:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62666:x1855007616745728:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074763.975907:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074763.975930:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd530d0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5a1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976065:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074763.976672:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 227 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976687:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976695:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976703:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 6/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074763.976741:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976756:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 15 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.976771:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625096576:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.976914:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976927:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 13 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.976935:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625096704:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074763.976986:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 227 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074763.976996:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 227 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:1.0:1769074763.977005:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625096832:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074763.977032:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074763.979986:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.981084:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745472 00000100:00100000:0.0:1769074763.981096:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745472 00002000:00100000:0.0:1769074763.981147:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (548864->552959) req@ffff89b546c6b480 x1855007616745472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.981159:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6b480 with x1855007616745472 ext(548864->552959) 00002000:00010000:0.0:1769074763.981165:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c6b480 00000100:00100000:0.0:1769074763.981228:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8379 00000100:00100000:0.0:1769074763.981237:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616745472:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074763.981435:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616745472:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 198us (9530us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074763.981445:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8379 00002000:00100000:0.0:1769074763.981464:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (548864->552959) req@ffff89b546c6b480 x1855007616745472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/4536 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074763.981473:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6b480 with x1855007616745472 ext(548864->552959) 00002000:00010000:0.0:1769074763.981476:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c6b480 00000100:00100000:3.0:1769074763.987394:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074763.987413:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074763.987435:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745856 00000100:00100000:3.0:1769074763.987439:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616745984 00000100:00100000:2.0:1769074763.987449:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745856 00000100:00100000:3.0:1769074763.987449:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616745984 00010000:00010000:3.0:1769074763.987501:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 16 pid: 6514 timeout: 1851 lvb_type: 0 00002000:00100000:2.0:1769074763.987514:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546c69c00 x1855007616745856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/0 e 0 to 0 dl 1769074774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074763.987527:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69c00 with x1855007616745856 ext(1048576->2097151) 00002000:00010000:2.0:1769074763.987533:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c69c00 00000100:00100000:3.0:1769074763.987537:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616746240 00000100:00100000:3.0:1769074763.987541:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616746240 00010000:00010000:3.0:1769074763.987555:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 228 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:2.0:1769074763.987560:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8380 00000100:00100000:2.0:1769074763.987576:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616745856:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074763.987585:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6876 00000100:00100000:3.0:1769074763.987594:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616745984:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.987941:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.987954:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 16 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.987970:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 16 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.987997:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074763.988002:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.988014:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.988021:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.988084:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616745984:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 490us (699us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.988090:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6876 00000100:00100000:3.0:1769074763.988109:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6877 00000100:00100000:3.0:1769074763.988117:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:2407:x1855007616746240:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074763.988126:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074763.988130:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 228 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.988144:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 228 pid: 6514 timeout: 1851 lvb_type: 0 00010000:00010000:3.0:1769074763.988157:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074763.988161:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.988179:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074763.988185:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074763.988204:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:2407:x1855007616746240:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 88us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074763.988210:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6877 00010000:00010000:0.0:1769074763.988521:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53083 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b570 expref: 15 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.988538:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625096576:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074763.988549:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625096704:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.988558:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd52ffe lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b50e expref: 227 pid: 6514 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.988566:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625096832:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074763.988589:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074763.992452:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616745344:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 30918us (33339us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074763.992480:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8378 00002000:00100000:1.0:1769074763.992576:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546c6a300 x1855007616745344/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074763.992588:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6a300 with x1855007616745344 ext(1048576->2097151) 00002000:00010000:1.0:1769074763.992600:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c6a300 00000100:00100000:2.0:1769074763.993029:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074763.993111:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616746112 00000100:00100000:0.0:1769074763.993119:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616746112 00010000:00010000:0.0:1769074763.993155:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 14 pid: 6512 timeout: 1851 lvb_type: 0 00000100:00100000:0.0:1769074763.993232:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6878 00000100:00100000:0.0:1769074763.993240:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2407:x1855007616746112:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074763.993258:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074763.993266:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 14 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:0.0:1769074763.993278:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 14 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:0.0:1769074763.993300:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5300c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b515 expref: 14 pid: 6512 timeout: 1851 lvb_type: 0 00010000:00010000:0.0:1769074763.993319:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074763.993325:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.993334:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.993342:0:6499:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074763.993373:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074763.993412:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616746112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 172us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074763.993418:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6878 00010000:00010000:2.0:1769074763.993476:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074763.993483:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074763.993502:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074763.993672:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd530d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5a1 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.993691:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5a1 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074763.993699:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074763.993701:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1769074763.993704:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4a00, rc 0) 00000100:00100000:2.0:1769074763.993739:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62666:x1855007616745728:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 17856us (21881us total) trans 8589943884 rc 0/0 00000100:00100000:2.0:1769074763.993747:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13282 00000100:00100000:0.0:1769074764.006696:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.006749:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.007137:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616746368 00000100:00100000:1.0:1769074764.007153:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616746368 00002000:00100000:1.0:1769074764.007220:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (913408->917503) req@ffff89b5446c2300 x1855007616746368/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.007235:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c2300 with x1855007616746368 ext(913408->917503) 00002000:00010000:1.0:1769074764.007244:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c2300 00000100:00100000:1.0:1769074764.007288:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8381 00000100:00100000:1.0:1769074764.007299:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616746368:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074764.007505:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616746368:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 207us (822us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074764.007515:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8381 00002000:00100000:1.0:1769074764.007522:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (913408->917503) req@ffff89b5446c2300 x1855007616746368/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/4536 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.007531:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c2300 with x1855007616746368 ext(913408->917503) 00002000:00010000:1.0:1769074764.007535:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c2300 00000100:00100000:2.0:1769074764.007790:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616745856:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20216us (27811us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074764.007798:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8380 00002000:00100000:2.0:1769074764.007830:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546c69c00 x1855007616745856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:299/0 lens 488/440 e 0 to 0 dl 1769074774 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.007839:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69c00 with x1855007616745856 ext(1048576->2097151) 00002000:00010000:2.0:1769074764.007843:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c69c00 00000100:00100000:3.0:1769074764.008227:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616746624 00000100:00100000:3.0:1769074764.008236:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616746624 00000100:00100000:3.0:1769074764.008301:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13283 00000100:00100000:3.0:1769074764.008310:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62666:x1855007616746624:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074764.008340:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074764.008344:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074764.008373:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd530d7 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5a1 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.008394:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.008401:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074764.008417:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd530e5 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5a8 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.008539:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008555:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008564:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008578:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008584:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008590:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008596:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.008599:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008606:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008614:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.008617:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074764.008621:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008625:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd530ec lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.008653:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd530e5 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5a8 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.008662:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.008664:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074764.008667:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd530e5 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5a8 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.008675:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:3.0:1769074764.008724:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62666:x1855007616746624:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 415us (1974us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074764.008730:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13283 00000100:00080000:2.0:1769074764.009690:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074764.019704:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.019886:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616746752 00000100:00100000:0.0:1769074764.019897:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616746752 00000100:00100000:0.0:1769074764.019959:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13861 00000100:00100000:0.0:1769074764.019981:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62657:x1855007616746752:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074764.020001:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.020024:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5af expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.020053:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.020055:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.020065:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.020087:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.020102:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cf6c600, rc 0) 00000100:00100000:0.0:1769074764.020181:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62657:x1855007616746752:12345-192.168.204.60@tcp:101:cat.0 Request processed in 212us (487us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.020189:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13861 00000100:00100000:2.0:1769074764.023467:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.023558:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.023663:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747264 00000100:00100000:1.0:1769074764.023675:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747264 00000100:00100000:1.0:1769074764.023737:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747008 00000100:00100000:1.0:1769074764.023741:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747008 00000100:00100000:1.0:1769074764.023757:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4637 00000100:00100000:1.0:1769074764.023766:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c5c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62666:x1855007616747264:12345-192.168.204.60@tcp:35:mirror_io.0 00000100:00100000:1.0:1769074764.024037:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c5c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62666:x1855007616747264:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 271us (591us total) trans 8589943885 rc 0/0 00000100:00100000:1.0:1769074764.024045:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4637 00000100:00100000:1.0:1769074764.024091:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4638 00000100:00100000:1.0:1769074764.024097:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62664:x1855007616747008:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074764.024179:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62664:x1855007616747008:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 81us (620us total) trans 8589943886 rc 0/0 00000100:00100000:1.0:1769074764.024186:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4638 00000100:00100000:2.0:1769074764.050420:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.050530:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747392 00000100:00100000:0.0:1769074764.050541:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747392 00002000:00100000:0.0:1769074764.050587:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54441c700 x1855007616747392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.050598:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c700 with x1855007616747392 ext(1048576->2097151) 00002000:00010000:0.0:1769074764.050603:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441c700 00000100:00100000:0.0:1769074764.050656:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8382 00000100:00100000:0.0:1769074764.050665:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616747392:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074764.065441:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616747392:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 14761us (15012us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074764.065465:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8382 00002000:00100000:2.0:1769074764.065479:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54441c700 x1855007616747392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.065490:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c700 with x1855007616747392 ext(1048576->2097151) 00002000:00010000:2.0:1769074764.065496:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441c700 00000100:00100000:2.0:1769074764.071267:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.072262:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747520 00000100:00100000:0.0:1769074764.072278:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747520 00002000:00100000:0.0:1769074764.072348:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54441c380 x1855007616747520/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.072360:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c380 with x1855007616747520 ext(2097152->3145727) 00002000:00010000:0.0:1769074764.072367:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441c380 00000100:00100000:0.0:1769074764.072475:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8383 00000100:00100000:0.0:1769074764.072485:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616747520:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074764.086297:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.086864:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747648 00000100:00100000:3.0:1769074764.086875:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747648 00002000:00100000:3.0:1769074764.086924:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b546f92300 x1855007616747648/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.086933:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92300 with x1855007616747648 ext(2097152->3145727) 00002000:00010000:3.0:1769074764.086940:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f92300 00000100:00100000:3.0:1769074764.086972:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8384 00000100:00100000:3.0:1769074764.086978:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616747648:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074764.098262:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616747520:12345-192.168.204.60@tcp:3:cat.0 Request processed in 25760us (26990us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074764.098280:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8383 00002000:00100000:0.0:1769074764.098297:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54441c380 x1855007616747520/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.098308:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441c380 with x1855007616747520 ext(2097152->3145727) 00002000:00010000:0.0:1769074764.098317:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441c380 00000100:00100000:3.0:1769074764.106406:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616747648:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19413us (20107us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074764.106425:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8384 00002000:00100000:3.0:1769074764.106442:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b546f92300 x1855007616747648/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.107156:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92300 with x1855007616747648 ext(2097152->3145727) 00002000:00010000:3.0:1769074764.107165:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f92300 00000100:00100000:2.0:1769074764.155136:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.155198:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.155235:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.155294:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747776 00000100:00100000:2.0:1769074764.155307:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747776 00000100:00100000:2.0:1769074764.155361:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616747904 00000100:00100000:2.0:1769074764.155365:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616747904 00000100:00100000:2.0:1769074764.155396:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13862 00000100:00100000:2.0:1769074764.155406:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616747776:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.155425:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.155444:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd530fa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5b6 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074764.155526:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625096960:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074764.155607:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00080000:2.0:1769074764.155641:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074764.155658:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13863 00000100:00100000:2.0:1769074764.155667:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616747904:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.155677:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.155687:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5bd expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.155700:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.155702:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.155712:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 28 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.155727:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 28 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.155739:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052e00, rc 0) 00000100:00100000:2.0:1769074764.155769:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616747904:12345-192.168.204.60@tcp:101:cat.0 Request processed in 103us (533us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.155776:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13863 00000100:00100000:3.0:1769074764.157204:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616748160 00000100:00100000:3.0:1769074764.157220:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616748160 00000100:00100000:3.0:1769074764.157317:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13284 00000100:00100000:3.0:1769074764.157328:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62670:x1855007616748160:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074764.157350:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074764.157373:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53108 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5c4 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.157464:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.157480:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.157491:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.157506:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.157513:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.157520:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.157973:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158001:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158010:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074764.158023:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074764.158129:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158136:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158141:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158147:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158153:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910d200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.158158:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158165:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158178:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074764.158181:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074764.158185:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158191:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5310f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158210:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158216:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158221:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158227:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.158230:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158235:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158242:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ead00) 00010000:00010000:3.0:1769074764.158244:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074764.158247:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.158253:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53116 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.159580:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53108 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 230 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.159604:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 229 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.159613:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.159616:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074764.159619:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:3.0:1769074764.159715:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62670:x1855007616748160:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 2389us (4515us total) trans 8589943887 rc 0/0 00000100:00100000:3.0:1769074764.159725:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13284 00000100:00080000:2.0:1769074764.161642:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:2.0:1769074764.172618:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074762 -> 1769074763 00002000:00010000:2.0:1769074764.172749:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074762 -> 1769074763 00002000:00010000:2.0:1769074764.172753:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074762 -> 1769074763 00000100:00100000:2.0:1769074764.172778:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625096960:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074764.173044:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd530fa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5b6 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.173068:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd530fa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5b6 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.173082:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:2.0:1769074764.173231:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616747776:12345-192.168.204.60@tcp:101:cat.0 Request processed in 17824us (18104us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074764.173242:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13862 00000100:00100000:2.0:1769074764.189176:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074764.189207:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625097088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074764.189296:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1769074764.189386:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074764.189718:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625097088 00000100:00100000:1.0:1769074764.189731:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625097088 00000100:00100000:1.0:1769074764.189828:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1156 00000100:00100000:1.0:1769074764.189840:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625097088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074764.189877:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 393216 granted 1012294976 pending 1699840 free 3900067840 avail 3668545536 00000100:00100000:1.0:1769074764.189942:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625097088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 105us (645us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.189950:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1156 00000100:00100000:1.0:1769074764.190010:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625097088:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074764.190214:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074764.190225:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625097216:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074764.190284:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074764.190315:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625097216 00000100:00100000:0.0:1769074764.190320:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625097216 00000100:00100000:0.0:1769074764.190350:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1157 00000100:00100000:0.0:1769074764.190356:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625097216:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074764.190372:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1032967488 pending 4853760 free 3905998848 avail 3674476544 00000100:00100000:0.0:1769074764.190404:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+21:3649:x1855007625097216:12345-0@lo:13:osp-pre-0-0.0 Request processed in 48us (119us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.190410:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1157 00000100:00100000:0.0:1769074764.190439:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625097216:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074764.191152:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074764.191170:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625097344:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074764.191210:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1769074764.191258:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074764.191262:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074764.191410:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625097344 00000100:00100000:1.0:1769074764.191417:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625097344 00000100:00100000:1.0:1769074764.191474:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1158 00000100:00100000:1.0:1769074764.191482:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625097344:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074764.191495:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1032967488 pending 4853760 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074764.191548:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625097344:12345-0@lo:13:osp-pre-0-1.0 Request processed in 67us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.191556:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1158 00000100:00100000:3.0:1769074764.191763:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625097344:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074764.205153:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.209782:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616748288 00000100:00100000:0.0:1769074764.210751:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616748288 00000100:00100000:2.0:1769074764.211066:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13864 00000100:00100000:2.0:1769074764.211079:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616748288:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.211112:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.211133:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5312b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5cb expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074764.211196:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625097472:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074764.211315:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.215203:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625097472:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074764.215265:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5312b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5cb expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.220766:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd5312b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5cb expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.220783:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:2.0:1769074764.220924:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616748288:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9845us (15779us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074764.220933:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13864 00000100:00100000:2.0:1769074764.232440:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.232550:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616748544 00000100:00100000:3.0:1769074764.232564:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616748544 00000100:00100000:3.0:1769074764.232655:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13285 00000100:00100000:3.0:1769074764.232666:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62673:x1855007616748544:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074764.232699:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074764.232727:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53132 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5d2 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.232879:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.232898:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.232909:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.232924:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.232931:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.232938:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233069:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233078:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233086:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074764.233095:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074764.233190:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233199:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233206:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233213:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233220:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910d200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.233224:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233233:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233247:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074764.233250:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074764.233255:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233266:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53139 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233280:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233285:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233290:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233295:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910da00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.233298:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233302:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233309:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ead00) 00010000:00010000:3.0:1769074764.233311:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074764.233313:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233318:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53140 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.233363:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd53132 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.233378:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 18 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.233390:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.233393:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074764.233396:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d800, rc 0) 00000100:00100000:3.0:1769074764.233481:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62673:x1855007616748544:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 817us (1055us total) trans 8589943888 rc 0/0 00000100:00100000:3.0:1769074764.233491:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13285 00000100:00100000:3.0:1769074764.241959:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.242062:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616748672 00000100:00100000:2.0:1769074764.242073:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616748672 00000100:00100000:2.0:1769074764.242133:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13865 00000100:00100000:2.0:1769074764.242144:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616748672:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.242162:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.242181:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd53155 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5d9 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074764.242239:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625097600:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074764.242337:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.255359:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625097600:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074764.255405:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd53155 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d9 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.255428:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053e00/0x7fe2323adfd53155 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d9 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.255442:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:2.0:1769074764.255681:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616748672:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13537us (13731us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074764.255691:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13865 00000100:00100000:2.0:1769074764.281462:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.282342:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616748800 00000100:00100000:0.0:1769074764.282359:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616748800 00000100:00100000:0.0:1769074764.283663:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13866 00000100:00100000:0.0:1769074764.283690:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62670:x1855007616748800:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074764.283717:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.283741:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5e0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.283776:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.283786:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074764.283819:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.283822:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.283827:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1380352->1892351] (req 1380352->1892351) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.283869:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074764.283884:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 1852 lvb_type: 1 00000100:00100000:0.0:1769074764.283899:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625097728:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074764.284015:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.284025:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 17 pid: 8398 timeout: 1852 lvb_type: 0 00000100:00100000:0.0:1769074764.284040:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625097856:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074764.284065:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074764.289121:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625097728:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074764.289143:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625097856:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074764.289194:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1380352->1892351] (req 1380352->1892351) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.289218:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cf6ce00, rc 0) 00000100:00100000:3.0:1769074764.289306:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62670:x1855007616748800:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 5628us (7867us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074764.289315:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13866 00000100:00100000:2.0:1769074764.296304:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.296392:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.296859:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616748928 00000100:00100000:1.0:1769074764.296871:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616748928 00010000:00010000:1.0:1769074764.296912:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 1852 lvb_type: 1 00000100:00100000:1.0:1769074764.296948:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749056 00000100:00100000:1.0:1769074764.296951:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749056 00010000:00010000:1.0:1769074764.296964:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 18 pid: 8398 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.297003:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6879 00000100:00100000:1.0:1769074764.297013:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616748928:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.297027:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.297048:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.297060:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.297082:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd53101 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5bd expref: 29 pid: 8398 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.297102:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.297154:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616748928:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 142us (863us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.297162:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6879 00000100:00100000:1.0:1769074764.297215:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6880 00000100:00100000:1.0:1769074764.297222:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616749056:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.297229:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.297236:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 18 pid: 8398 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.297246:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 18 pid: 8398 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.297264:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd53067 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b569 expref: 18 pid: 8398 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.297289:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.297291:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.297298:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 27 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.297307:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 27 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.297341:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 27 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074764.297355:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625097984:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074764.297403:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.310937:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.312086:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749184 00000100:00100000:3.0:1769074764.312101:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749184 00000100:00100000:2.0:1769074764.312539:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13867 00000100:00100000:2.0:1769074764.312556:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62673:x1855007616749184:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074764.312578:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.312602:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5e7 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074764.313753:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625097984:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074764.313778:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074764.313862:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616749056:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16639us (17468us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.313871:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6880 00000100:00100000:3.0:1769074764.314272:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.314582:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749312 00000100:00100000:3.0:1769074764.314589:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749312 00002000:00100000:3.0:1769074764.314778:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1380352->1384447) req@ffff89b546c6bb80 x1855007616749312/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.314789:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6bb80 with x1855007616749312 ext(1380352->1384447) 00002000:00010000:3.0:1769074764.314942:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c6bb80 00010000:00010000:2.0:1769074764.315049:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.315061:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 9 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.315069:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.315077:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.315079:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.315083:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.315120:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.315135:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074764.315149:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098112:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074764.315235:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 9 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.315244:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 9 pid: 9048 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.315252:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098240:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074764.315285:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.315293:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074764.315301:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098368:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.315320:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074764.319061:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8385 00000100:00100000:1.0:1769074764.319086:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616749312:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074764.319279:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616749312:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 205us (5007us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074764.319288:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8385 00002000:00100000:1.0:1769074764.319298:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1380352->1384447) req@ffff89b546c6bb80 x1855007616749312/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/4536 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.319308:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6bb80 with x1855007616749312 ext(1380352->1384447) 00002000:00010000:1.0:1769074764.319314:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c6bb80 00000100:00100000:2.0:1769074764.329979:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098112:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.329998:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098240:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.330006:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098368:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074764.330040:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.330063:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:2.0:1769074764.330131:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62673:x1855007616749184:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17579us (19204us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.330140:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13867 00000100:00100000:3.0:1769074764.331943:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.331984:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.332014:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.332303:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749568 00000100:00100000:1.0:1769074764.332316:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749568 00010000:00010000:1.0:1769074764.332380:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 10 pid: 9048 timeout: 1852 lvb_type: 1 00000100:00100000:1.0:1769074764.332411:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749440 00000100:00100000:1.0:1769074764.332415:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749440 00010000:00010000:1.0:1769074764.332428:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.332442:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749696 00000100:00100000:1.0:1769074764.332444:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749696 00010000:00010000:1.0:1769074764.332468:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.332487:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6881 00000100:00100000:1.0:1769074764.332496:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616749568:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.332513:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.332539:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 10 pid: 9048 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.332551:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 10 pid: 9048 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.332576:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd530b4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b585 expref: 10 pid: 9048 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.332602:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.332674:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616749568:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 178us (740us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.332682:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6881 00000100:00100000:1.0:1769074764.332702:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6882 00000100:00100000:1.0:1769074764.332709:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616749440:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.332716:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.332724:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.332735:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.332748:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53060 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b562 expref: 28 pid: 10284 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.332757:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.332776:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616749440:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 68us (792us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.332783:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6882 00000100:00100000:1.0:1769074764.332841:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6883 00000100:00100000:1.0:1769074764.332849:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616749696:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.332858:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.332867:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.332878:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.332892:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53059 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b55b expref: 18 pid: 10284 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.332902:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.332904:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.332911:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.332921:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.332957:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074764.332971:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625098496:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074764.333021:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00080000:3.0:1769074764.333667:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074764.339122:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.339217:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749824 00000100:00100000:1.0:1769074764.339227:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749824 00002000:00100000:1.0:1769074764.339279:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1888256->1892351) req@ffff89b546c68a80 x1855007616749824/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.339290:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c68a80 with x1855007616749824 ext(1888256->1892351) 00002000:00010000:1.0:1769074764.339297:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c68a80 00000100:00100000:1.0:1769074764.339327:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8386 00000100:00100000:1.0:1769074764.339335:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616749824:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074764.339507:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616749824:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 172us (394us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074764.339515:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8386 00002000:00100000:1.0:1769074764.339521:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1888256->1892351) req@ffff89b546c68a80 x1855007616749824/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/4536 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.339529:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c68a80 with x1855007616749824 ext(1888256->1892351) 00002000:00010000:1.0:1769074764.339532:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c68a80 00000100:00100000:1.0:1769074764.345414:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625098496:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074764.345458:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.345531:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616749696:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12681us (13513us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.345541:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6883 00000100:00100000:3.0:1769074764.346026:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.346317:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616749952 00000100:00100000:0.0:1769074764.346334:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616749952 00002000:00100000:0.0:1769074764.346394:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1052671) req@ffff89b546c68e00 x1855007616749952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.346406:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c68e00 with x1855007616749952 ext(1048576->1052671) 00002000:00010000:0.0:1769074764.346413:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c68e00 00000100:00100000:0.0:1769074764.346446:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8387 00000100:00100000:0.0:1769074764.346467:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616749952:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074764.346745:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616749952:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 280us (719us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074764.346753:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8387 00002000:00100000:0.0:1769074764.346760:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1052671) req@ffff89b546c68e00 x1855007616749952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/4536 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.346769:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c68e00 with x1855007616749952 ext(1048576->1052671) 00002000:00010000:0.0:1769074764.346773:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c68e00 00000100:00100000:3.0:1769074764.353970:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.354506:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616750208 00000100:00100000:2.0:1769074764.354519:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616750208 00000100:00100000:2.0:1769074764.354581:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4639 00000100:00100000:2.0:1769074764.354591:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62670:x1855007616750208:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074764.354924:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:62670:x1855007616750208:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 333us (963us total) trans 8589943889 rc 0/0 00000100:00100000:2.0:1769074764.354937:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4639 00000100:00100000:2.0:1769074764.366040:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.366139:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616750464 00000100:00100000:3.0:1769074764.366150:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616750464 00000100:00100000:3.0:1769074764.366212:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4640 00000100:00100000:3.0:1769074764.366222:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62673:x1855007616750464:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074764.366499:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62673:x1855007616750464:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 277us (468us total) trans 8589943890 rc 0/0 00000100:00100000:3.0:1769074764.366508:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4640 00000100:00100000:0.0:1769074764.437337:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.437461:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616750592 00000100:00100000:2.0:1769074764.437472:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616750592 00000100:00100000:2.0:1769074764.437534:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13868 00000100:00100000:2.0:1769074764.437544:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616750592:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.437562:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.437580:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd5316a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5ee expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074764.437641:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098624:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074764.437738:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.441105:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098624:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074764.441148:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd5316a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5ee expref: 18 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.441359:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd5316a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5ee expref: 18 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.441374:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:2.0:1769074764.441760:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616750592:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4215us (4429us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074764.441770:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13868 00000100:00100000:0.0:1769074764.454710:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.454780:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.455070:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616750720 00000100:00100000:2.0:1769074764.455085:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616750720 00002000:00100000:2.0:1769074764.455150:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->548863) req@ffff89b5446c3800 x1855007616750720/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.455163:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c3800 with x1855007616750720 ext(0->548863) 00002000:00010000:2.0:1769074764.455170:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c3800 00000100:00100000:2.0:1769074764.455200:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616750848 00000100:00100000:2.0:1769074764.455204:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616750848 00002000:00100000:2.0:1769074764.455222:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1052672->2097151) req@ffff89b5446c1500 x1855007616750848/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.455232:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1500 with x1855007616750848 ext(1052672->2097151) 00002000:00010000:2.0:1769074764.455236:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c1500 00000100:00100000:2.0:1769074764.455277:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8388 00000100:00100000:2.0:1769074764.455288:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616750720:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074764.456152:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8389 00000100:00100000:3.0:1769074764.456166:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616750848:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074764.471959:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616750720:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16657us (17244us total) trans 0 rc 548864/548864 00000100:00100000:2.0:1769074764.471978:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8388 00002000:00100000:2.0:1769074764.471998:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->548863) req@ffff89b5446c3800 x1855007616750720/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 548864/548864 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.472011:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c3800 with x1855007616750720 ext(0->548863) 00002000:00010000:2.0:1769074764.472018:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c3800 00000100:00100000:2.0:1769074764.510497:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616750848:12345-192.168.204.60@tcp:3:cat.0 Request processed in 54270us (55652us total) trans 0 rc 1044480/1044480 00000100:00100000:2.0:1769074764.510511:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8389 00002000:00100000:2.0:1769074764.510525:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1052672->2097151) req@ffff89b5446c1500 x1855007616750848/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1044480/1044480 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.510542:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c1500 with x1855007616750848 ext(1052672->2097151) 00002000:00010000:2.0:1769074764.510548:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5446c1500 00000100:00100000:3.0:1769074764.531080:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.532075:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616750976 00000100:00100000:2.0:1769074764.532090:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616750976 00000100:00100000:2.0:1769074764.532167:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13869 00000100:00100000:2.0:1769074764.532178:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62680:x1855007616750976:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.532196:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.532218:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5f5 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.532258:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 26 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.532279:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074764.532281:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074764.532286:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.532324:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 26 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.532338:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 26 pid: 9048 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074764.532352:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098752:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.533152:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.542891:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.544359:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616751104 00000100:00100000:1.0:1769074764.544375:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616751104 00002000:00100000:1.0:1769074764.544461:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1380352->1892351) req@ffff89b54759c380 x1855007616751104/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.544474:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759c380 with x1855007616751104 ext(1380352->1892351) 00010000:00010000:1.0:1769074764.544484:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 27 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.544503:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 27 pid: 9048 timeout: 1852 lvb_type: 0 00002000:00010000:1.0:1769074764.544519:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54759c380 00000100:00100000:1.0:1769074764.544565:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 711 00000100:00100000:1.0:1769074764.544576:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616751104:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074764.548126:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625098752:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074764.548353:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.548378:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:0.0:1769074764.548785:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62680:x1855007616750976:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16606us (17713us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.550403:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13869 00000100:00100000:1.0:1769074764.569783:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616751104:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 25194us (26887us total) trans 21474841173 rc 0/0 00000100:00100000:1.0:1769074764.569800:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 711 00002000:00100000:1.0:1769074764.569843:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1380352->1892351) req@ffff89b54759c380 x1855007616751104/t21474841173(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/448 e 0 to 0 dl 1769074775 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.569854:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759c380 with x1855007616751104 ext(1380352->1892351) 00010000:00010000:1.0:1769074764.569863:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.569882:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 1852 lvb_type: 0 00002000:00010000:1.0:1769074764.569890:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54759c380 00000100:00100000:2.0:1769074764.578474:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.578647:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616751232 00000100:00100000:1.0:1769074764.578659:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616751232 00010000:00010000:1.0:1769074764.578700:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.578742:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6884 00000100:00100000:1.0:1769074764.578752:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616751232:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.578770:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.578802:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.580349:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.580378:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cf6ce00/0x7fe2323adfd5315c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1892351) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e0 expref: 29 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.580398:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.580400:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.580408:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.580418:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.580486:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074764.580502:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625098880:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074764.580600:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.583171:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.583278:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.586440:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.587526:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616751360 00000100:00100000:0.0:1769074764.587541:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616751360 00000100:00100000:0.0:1769074764.587647:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616751616 00000100:00100000:0.0:1769074764.587651:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616751616 00000100:00100000:0.0:1769074764.587692:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13870 00000100:00100000:0.0:1769074764.587701:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616751360:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074764.587719:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.587736:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53178 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b5fc expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074764.587804:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625099008:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074764.587905:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656040c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074764.587932:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13871 00000100:00100000:0.0:1769074764.587940:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616751616:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074764.587948:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.587958:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b60a expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.587972:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.587974:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.587984:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 30 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074764.587999:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 30 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074764.588011:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97400, rc 0) 00000100:00100000:0.0:1769074764.588038:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616751616:12345-192.168.204.60@tcp:101:cat.0 Request processed in 99us (4760us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.588046:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13871 00000100:00100000:2.0:1769074764.602644:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.602975:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616751488 00000100:00100000:3.0:1769074764.602988:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616751488 00000100:00100000:3.0:1769074764.603057:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13872 00000100:00100000:3.0:1769074764.603067:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616751488:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074764.603085:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074764.603103:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53186 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b603 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074764.603182:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625099136:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074764.603610:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074764.603661:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616751744 00000100:00100000:3.0:1769074764.603667:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616751744 00002000:00100000:3.0:1769074764.603703:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e6a00 x1855007616751744/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.603713:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6a00 with x1855007616751744 ext(1048576->2097151) 00002000:00010000:3.0:1769074764.603718:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e6a00 00000100:00100000:3.0:1769074764.603737:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8390 00000100:00100000:3.0:1769074764.603744:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616751744:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074764.604590:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625098880:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074764.604624:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074764.604666:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616751232:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 25915us (26202us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.604674:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6884 00002000:00010000:2.0:1769074764.615907:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074763 -> 1769074764 00000100:00100000:2.0:1769074764.616727:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625099008:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074764.616776:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53178 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5fc expref: 23 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.616838:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53178 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5fc expref: 23 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.616853:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d96e00, rc 0) 00000100:00100000:3.0:1769074764.618759:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.619292:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616751360:12345-192.168.204.60@tcp:101:cat.0 Request processed in 31589us (36135us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074764.619306:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13870 00000100:00100000:1.0:1769074764.622505:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752000 00000100:00100000:1.0:1769074764.622527:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752000 00000100:00100000:1.0:1769074764.622649:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13286 00000100:00100000:1.0:1769074764.622663:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:62685:x1855007616752000:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074764.622703:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.622738:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd5318d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b611 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.622904:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.622929:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 231 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.622938:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 18 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.622946:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 5/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.622986:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 18 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.622999:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 18 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.623012:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625099264:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074764.623113:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 231 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.623121:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 231 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.623134:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625099392:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074764.623180:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb480 going to sleep for 11 seconds 00002000:00010000:3.0:1769074764.625366:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074762 -> 1769074764 00000100:00100000:3.0:1769074764.625395:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625099136:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074764.625431:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53186 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b603 expref: 30 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074764.625461:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53186 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b603 expref: 30 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074764.625474:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074764.625524:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616751488:12345-192.168.204.60@tcp:101:cat.0 Request processed in 22457us (39095us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074764.625533:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13872 00000100:00100000:2.0:1769074764.635112:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.635128:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625099264:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074764.635143:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625099392:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.635145:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074764.635169:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:3.0:1769074764.636837:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752128 00000100:00100000:3.0:1769074764.636853:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752128 00010000:00010000:3.0:1769074764.636900:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 19 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074764.636947:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752256 00000100:00100000:3.0:1769074764.636951:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752256 00010000:00010000:3.0:1769074764.636966:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 232 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074764.636991:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6885 00000100:00100000:3.0:1769074764.636999:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616752128:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074764.637017:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074764.637024:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 19 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.637036:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 19 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.637061:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53147 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5d2 expref: 19 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.637078:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.637083:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.637092:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.637098:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.637119:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6886 00000100:00100000:1.0:1769074764.637128:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+232:2408:x1855007616752256:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.637136:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.637141:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 232 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.637150:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 232 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.637164:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5311d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5c4 expref: 232 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.637174:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00000100:00100000:3.0:1769074764.637174:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616752128:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 175us (2074us total) trans 0 rc 0/0 00010000:00010000:1.0:1769074764.637178:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:3.0:1769074764.637183:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6885 00010000:00010000:1.0:1769074764.637188:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.637199:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.637228:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.637250:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:2408:x1855007616752256:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 123us (2103us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.637257:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6886 00010000:00010000:1.0:1769074764.637315:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.637323:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074764.637344:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074764.637495:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd5318d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 232 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.637509:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 231 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.637517:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074764.637519:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074764.637521:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4000, rc 0) 00000100:00100000:1.0:1769074764.637567:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62685:x1855007616752000:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 14908us (18819us total) trans 8589943891 rc 0/0 00000100:00100000:1.0:1769074764.637583:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13286 00000100:00100000:1.0:1769074764.640841:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616751744:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37058us (38169us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074764.640861:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8390 00002000:00100000:1.0:1769074764.640877:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e6a00 x1855007616751744/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.640887:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6a00 with x1855007616751744 ext(1048576->2097151) 00002000:00010000:1.0:1769074764.640894:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e6a00 00000100:00100000:3.0:1769074764.643469:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.643549:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752384 00000100:00100000:0.0:1769074764.643559:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752384 00000100:00100000:0.0:1769074764.643628:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13873 00000100:00100000:0.0:1769074764.643638:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62683:x1855007616752384:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074764.643658:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.643679:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b618 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.643717:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.643726:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074764.643728:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074764.643732:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.643771:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.643785:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 14 pid: 8396 timeout: 1852 lvb_type: 0 00000100:00100000:0.0:1769074764.644058:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625099520:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074764.644148:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074764.660092:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625099520:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074764.660132:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.660152:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97800, rc 0) 00000100:00100000:0.0:1769074764.660239:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62683:x1855007616752384:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16602us (16779us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.660250:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13873 00000100:00100000:2.0:1769074764.665224:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.665473:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752768 00000100:00100000:1.0:1769074764.665484:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752768 00000100:00100000:1.0:1769074764.665543:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13287 00000100:00100000:1.0:1769074764.665553:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62687:x1855007616752768:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074764.665576:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.665596:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd531a9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b61f expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.665757:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.665771:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.665782:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.666036:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.666045:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.666052:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.669981:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670009:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670023:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074764.670036:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074764.670189:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670198:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670205:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670214:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670220:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074764.670225:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670235:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670251:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074764.670255:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074764.670278:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670288:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd531b0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670309:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670315:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670321:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670324:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea5400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074764.670326:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670332:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670339:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615b00) 00010000:00010000:1.0:1769074764.670342:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074764.670345:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670350:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd531b7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.670409:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd531a9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 21 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.670424:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 20 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.670436:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074764.670439:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074764.670451:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f71a00, rc 0) 00000100:00100000:1.0:1769074764.670550:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62687:x1855007616752768:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 4997us (5335us total) trans 8589943892 rc 0/0 00000100:00100000:1.0:1769074764.670561:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13287 00000100:00100000:2.0:1769074764.674114:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.676409:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.679426:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752512 00000100:00100000:1.0:1769074764.679453:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752512 00002000:00100000:1.0:1769074764.679517:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (548864->1048575) req@ffff89b54d0e6d80 x1855007616752512/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.679527:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6d80 with x1855007616752512 ext(548864->1048575) 00010000:00010000:1.0:1769074764.679536:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 15 pid: 8396 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074764.679552:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 15 pid: 8396 timeout: 1852 lvb_type: 0 00002000:00010000:1.0:1769074764.679560:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d0e6d80 00000100:00100000:1.0:1769074764.679585:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 712 00000100:00100000:1.0:1769074764.679594:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616752512:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074764.687684:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616752896 00000100:00100000:0.0:1769074764.687708:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616752896 00000100:00100000:2.0:1769074764.689683:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13874 00000100:00100000:2.0:1769074764.689696:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616752896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.689718:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.689741:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531cc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b626 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074764.689800:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625099648:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074764.689914:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074764.691406:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.691495:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753024 00000100:00100000:1.0:1769074764.691506:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753024 00000100:00100000:1.0:1769074764.691557:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13288 00000100:00100000:1.0:1769074764.691565:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62688:x1855007616753024:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074764.691587:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.691615:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70400/0x7fe2323adfd531d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b62d expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.691679:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691693:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691703:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691715:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691721:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691727:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691797:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691834:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 230 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.691842:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.691869:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 230 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.691885:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 230 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.691898:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625099776:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074764.691995:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8b40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074764.703833:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074764.703861:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625099904:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074764.703925:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074764.703972:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625099904 00000100:00100000:0.0:1769074764.703980:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625099904 00000100:00100000:0.0:1769074764.704026:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1159 00000100:00100000:0.0:1769074764.704034:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625099904:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074764.704057:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1020732736 pending 2224128 free 3900067840 avail 3668545536 00000100:00100000:0.0:1769074764.704099:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625099904:12345-0@lo:13:osp-pre-1-1.0 Request processed in 66us (173us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.704106:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1159 00000100:00100000:0.0:1769074764.704154:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625099904:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074764.706245:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625099648:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074764.706312:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531cc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b626 expref: 20 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.706349:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531cc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b626 expref: 20 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.706366:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47600, rc 0) 00000100:00100000:2.0:1769074764.706456:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616752896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16762us (30057us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074764.706466:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13874 00000100:00100000:3.0:1769074764.709201:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616752512:12345-192.168.204.60@tcp:4:cat.0 Request processed in 29569us (35064us total) trans 17179874491 rc 0/0 00000100:00100000:3.0:1769074764.709224:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 712 00002000:00100000:3.0:1769074764.709245:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (548864->1048575) req@ffff89b54d0e6d80 x1855007616752512/t17179874491(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/448 e 0 to 0 dl 1769074775 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.709258:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6d80 with x1855007616752512 ext(548864->1048575) 00010000:00010000:3.0:1769074764.710004:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.710026:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 1852 lvb_type: 0 00002000:00010000:3.0:1769074764.710037:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d0e6d80 00000100:00100000:0.0:1769074764.721933:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.721971:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.722011:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.722256:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753152 00000100:00100000:3.0:1769074764.722293:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753152 00010000:00010000:3.0:1769074764.722354:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 231 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074764.722412:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753280 00000100:00100000:3.0:1769074764.722418:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753280 00010000:00010000:3.0:1769074764.722452:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074764.722480:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6887 00000100:00100000:3.0:1769074764.722492:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:2408:x1855007616753152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074764.722510:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074764.722519:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 231 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.722535:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 231 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.722563:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074764.722570:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.722583:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.722593:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074764.722619:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074764.722692:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:2408:x1855007616753152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 200us (779us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074764.722701:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6887 00000100:00100000:3.0:1769074764.722731:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6888 00000100:00100000:3.0:1769074764.722741:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616753280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074764.722753:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074764.722782:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.722796:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.722869:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd530c9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 548864->917503) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b59a expref: 17 pid: 8396 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074764.722889:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074764.722893:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074764.722902:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.722913:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.722963:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074764.722977:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625100032:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074764.723038:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074764.724868:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753408 00000100:00100000:2.0:1769074764.724886:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753408 00000100:00100000:2.0:1769074764.724967:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13875 00000100:00100000:2.0:1769074764.724978:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616753408:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.724995:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.725015:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b634 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.725038:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.725040:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.725050:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 18 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.725066:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 18 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.725079:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47600, rc 0) 00000100:00100000:2.0:1769074764.725187:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616753408:12345-192.168.204.60@tcp:101:cat.0 Request processed in 208us (3171us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.725197:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13875 00010000:00010000:1.0:1769074764.725855:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53194 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b611 expref: 230 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.725874:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625099776:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074764.725896:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.725904:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.725909:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074764.725971:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074764.725975:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074764.725994:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726008:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726021:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726027:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726032:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726039:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726044:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f70000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074764.726048:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726056:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726069:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074764.726073:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074764.726078:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726084:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70000/0x7fe2323adfd531da lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726095:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726100:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726105:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726110:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f71400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074764.726112:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726121:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726126:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074764.726128:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074764.726130:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726135:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd531e1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074764.726179:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70400/0x7fe2323adfd531d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.726192:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.726200:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074764.726202:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074764.726205:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3400, rc 0) 00000100:00100000:1.0:1769074764.726256:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62688:x1855007616753024:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 34692us (34859us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.726469:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13288 00000100:00100000:1.0:1769074764.753029:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.755100:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753536 00000100:00100000:0.0:1769074764.755120:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753536 00002000:00100000:0.0:1769074764.755193:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b546f93800 x1855007616753536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.755206:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93800 with x1855007616753536 ext(0->1048575) 00002000:00010000:0.0:1769074764.755213:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f93800 00000100:00100000:0.0:1769074764.755281:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8391 00000100:00100000:0.0:1769074764.755296:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616753536:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074764.762723:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625100032:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074764.762754:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074764.763058:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616753280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 40316us (41083us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.763070:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6888 00000100:00100000:1.0:1769074764.776263:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.776296:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.776325:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.776358:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753664 00000100:00100000:1.0:1769074764.776372:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753664 00000100:00100000:1.0:1769074764.776457:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753920 00000100:00100000:1.0:1769074764.776461:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753920 00000100:00100000:1.0:1769074764.776480:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616753792 00000100:00100000:1.0:1769074764.776483:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616753792 00000100:00100000:1.0:1769074764.776502:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13876 00000100:00100000:1.0:1769074764.776511:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616753664:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074764.776532:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.776552:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b63b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.776576:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.776579:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.776590:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074764.776622:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074764.776638:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:1.0:1769074764.776706:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616753664:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 195us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.776714:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13876 00000100:00100000:1.0:1769074764.776736:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13877 00000100:00100000:1.0:1769074764.776742:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616753920:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074764.776749:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.776760:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b649 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.776782:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.776784:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.776790:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074764.776801:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074764.776844:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:1.0:1769074764.776869:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616753920:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 128us (573us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.776875:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13877 00000100:00100000:1.0:1769074764.776887:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13878 00000100:00100000:1.0:1769074764.776898:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616753792:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074764.776908:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.777278:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53204 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b642 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074764.777343:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625100160:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074764.777416:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074764.777509:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.777544:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754048 00000100:00100000:1.0:1769074764.777552:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754048 00000100:00100000:1.0:1769074764.777586:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13879 00000100:00100000:1.0:1769074764.777593:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616754048:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074764.777674:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074764.777687:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b650 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074764.777699:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.777701:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074764.777708:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 12 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074764.777720:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 12 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074764.777729:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:1.0:1769074764.777755:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616754048:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 163us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.777762:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13879 00000100:00080000:3.0:1769074764.778082:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074764.795894:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616753536:12345-192.168.204.60@tcp:3:cat.0 Request processed in 40588us (42864us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074764.795911:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8391 00002000:00100000:3.0:1769074764.796285:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b546f93800 x1855007616753536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.796297:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93800 with x1855007616753536 ext(0->1048575) 00002000:00010000:3.0:1769074764.796303:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f93800 00000100:00100000:3.0:1769074764.800575:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625100160:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074764.800638:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53204 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b642 expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074764.800661:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53204 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b642 expref: 11 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074764.800674:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:3.0:1769074764.800747:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616753792:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 23851us (24419us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074764.800756:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13878 00000100:00100000:1.0:1769074764.816409:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.816530:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.816967:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754432 00000100:00100000:0.0:1769074764.816977:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754432 00000100:00100000:0.0:1769074764.817051:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13289 00000100:00100000:0.0:1769074764.817061:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62688:x1855007616754432:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074764.817083:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.817108:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd53212 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b65e expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.817483:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.817503:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.817513:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.817549:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.817564:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 15 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:0.0:1769074764.817580:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625100288:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.818141:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754176 00000100:00100000:2.0:1769074764.818150:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754176 00000100:00100000:2.0:1769074764.818207:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13880 00000100:00100000:2.0:1769074764.818215:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62687:x1855007616754176:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074764.818228:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.818243:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b657 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.818268:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.818277:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 11 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.818286:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.818294:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.818296:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.818299:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2666496->3137535] (req 2666496->3137535) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.818321:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.818331:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 22 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.818342:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625100416:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074764.818415:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00010000:00010000:2.0:1769074764.820520:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 11 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.820539:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 11 pid: 28954 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.820556:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625100544:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074764.822332:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074764.822353:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.822374:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625100672:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.822421:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074764.825570:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.826131:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625100288:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074764.826181:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:2.0:1769074764.826420:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754560 00000100:00100000:2.0:1769074764.826429:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754560 00010000:00010000:2.0:1769074764.826486:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 16 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074764.826520:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6889 00000100:00100000:2.0:1769074764.826529:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2406:x1855007616754560:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074764.826548:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074764.826555:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 16 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074764.826566:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 16 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074764.826588:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd531e8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b62d expref: 16 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074764.826617:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074764.826622:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074764.826632:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074764.826639:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074764.826668:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074764.826721:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2406:x1855007616754560:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 192us (1164us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.826728:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6889 00010000:00010000:0.0:1769074764.827587:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.827608:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074764.827634:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074764.827846:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd53212 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b65e expref: 16 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.827864:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b65e expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.827873:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074764.827876:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1769074764.827880:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:0.0:1769074764.828088:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62688:x1855007616754432:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 11028us (11557us total) trans 8589943893 rc 0/0 00000100:00100000:0.0:1769074764.828098:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13289 00000100:00100000:2.0:1769074764.836032:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625100416:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.836053:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625100544:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.838423:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625100672:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074764.838482:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2666496->3137535] (req 2666496->3137535) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.838509:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47400, rc 0) 00000100:00100000:2.0:1769074764.838593:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62687:x1855007616754176:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20377us (22194us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.842088:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13880 00000100:00100000:1.0:1769074764.843276:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.843569:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754688 00000100:00100000:2.0:1769074764.843578:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754688 00000100:00100000:2.0:1769074764.843642:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13881 00000100:00100000:2.0:1769074764.843651:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62683:x1855007616754688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074764.843667:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074764.843682:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b665 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.843710:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.843720:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074764.843722:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074764.843725:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 30 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.843750:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.843758:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 17 pid: 9048 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074764.843767:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625100800:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074764.843849:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074764.853182:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.853379:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616755200 00000100:00100000:0.0:1769074764.853391:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616755200 00000100:00100000:0.0:1769074764.853465:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13290 00000100:00100000:0.0:1769074764.853476:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62688:x1855007616755200:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074764.853506:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074764.853510:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1769074764.853536:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53219 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b65e expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.853555:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074764.853560:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074764.853574:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd53235 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b66c expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.853686:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853701:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853710:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853723:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853729:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853735:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853741:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ee00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074764.853745:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853753:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853761:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074764.853764:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074764.853767:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853773:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5323c lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074764.853861:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd53235 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b66c expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.853870:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074764.853872:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074764.853876:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd53235 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b66c expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.853884:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fc00, rc 0) 00000100:00100000:0.0:1769074764.854039:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62688:x1855007616755200:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 565us (868us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074764.854047:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13290 00000100:00100000:1.0:1769074764.858520:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.858564:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.858603:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.858681:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754816 00000100:00100000:2.0:1769074764.858691:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754816 00010000:00010000:2.0:1769074764.858743:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 12 pid: 28954 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.858789:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616754944 00000100:00100000:2.0:1769074764.858793:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616754944 00010000:00010000:2.0:1769074764.858834:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.858859:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616755328 00000100:00100000:2.0:1769074764.858862:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616755328 00010000:00010000:2.0:1769074764.858877:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 23 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074764.858903:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6890 00000100:00100000:2.0:1769074764.858912:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616754816:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1769074764.859028:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6891 00000100:00100000:1.0:1769074764.859038:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616754944:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.859057:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.859080:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.859092:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.859117:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6da00/0x7fe2323adfd52dab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b39b expref: 18 pid: 36516 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.859135:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.859203:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616754944:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 167us (637us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.859211:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6891 00000100:00100000:1.0:1769074764.859231:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6892 00000100:00100000:1.0:1769074764.859237:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616755328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074764.859245:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074764.859254:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 23 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.859265:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 23 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.859279:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ec00/0x7fe2323adfd52de3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3c5 expref: 23 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074764.859289:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074764.859308:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616755328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 71us (713us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074764.859315:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6892 00010000:00010000:2.0:1769074764.859606:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074764.859617:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 12 pid: 28954 timeout: 1852 lvb_type: 1 00010000:00010000:2.0:1769074764.859628:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 12 pid: 28954 timeout: 1852 lvb_type: 1 00010000:00010000:2.0:1769074764.859643:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd52e7d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b427 expref: 12 pid: 28954 timeout: 1852 lvb_type: 1 00010000:00010000:2.0:1769074764.859653:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.859655:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074764.859662:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.859673:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.859717:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074764.859737:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625100928:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074764.859888:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074764.861985:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074764.862138:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616755456 00000100:00100000:0.0:1769074764.862151:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616755456 00002000:00100000:0.0:1769074764.862226:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1052671) req@ffff89b54441dc00 x1855007616755456/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 4584/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.862237:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441dc00 with x1855007616755456 ext(1048576->1052671) 00010000:00010000:0.0:1769074764.862248:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 18 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:0.0:1769074764.862264:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 18 pid: 9048 timeout: 1852 lvb_type: 0 00002000:00010000:0.0:1769074764.862272:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441dc00 00000100:00100000:0.0:1769074764.862307:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 713 00000100:00100000:0.0:1769074764.862316:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616755456:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074764.864735:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616755456:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 2408us (2753us total) trans 21474841174 rc 0/0 00000100:00100000:0.0:1769074764.864751:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 713 00002000:00100000:0.0:1769074764.864764:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1052671) req@ffff89b54441dc00 x1855007616755456/t21474841174(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 4584/448 e 0 to 0 dl 1769074775 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.864777:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441dc00 with x1855007616755456 ext(1048576->1052671) 00010000:00010000:0.0:1769074764.864797:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:0.0:1769074764.864843:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00002000:00010000:0.0:1769074764.864851:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441dc00 00000100:00100000:2.0:1769074764.866853:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625100800:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074764.866890:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 30 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.866908:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:2.0:1769074764.866985:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:62683:x1855007616754688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 23333us (23714us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.866994:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13881 00000100:00100000:3.0:1769074764.875494:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.875631:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625100928:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074764.875671:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074764.875751:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616754816:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16837us (17240us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074764.875764:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6890 00000100:00100000:0.0:1769074764.876956:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616755584 00000100:00100000:0.0:1769074764.876969:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616755584 00002000:00100000:0.0:1769074764.877024:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2666496->2670591) req@ffff89b546c6ad80 x1855007616755584/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.877036:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6ad80 with x1855007616755584 ext(2666496->2670591) 00002000:00010000:0.0:1769074764.877041:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c6ad80 00000100:00100000:0.0:1769074764.877070:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8392 00000100:00100000:0.0:1769074764.877080:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616755584:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074764.877249:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616755584:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 169us (1765us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074764.877668:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8392 00002000:00100000:0.0:1769074764.877681:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2666496->2670591) req@ffff89b546c6ad80 x1855007616755584/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/4536 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074764.877690:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6ad80 with x1855007616755584 ext(2666496->2670591) 00002000:00010000:0.0:1769074764.877693:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c6ad80 00000100:00100000:3.0:1769074764.885362:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.885452:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.885799:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616755712 00000100:00100000:2.0:1769074764.885849:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616755712 00010000:00010000:2.0:1769074764.885896:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074764.886136:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616755968 00000100:00100000:1.0:1769074764.886142:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616755968 00000100:00100000:1.0:1769074764.886201:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4641 00000100:00100000:1.0:1769074764.886213:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62688:x1855007616755968:12345-192.168.204.60@tcp:35:mirror_io.0 00000100:00100000:0.0:1769074764.886376:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6893 00000100:00100000:0.0:1769074764.886386:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616755712:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074764.886402:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074764.886429:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:0.0:1769074764.886455:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:0.0:1769074764.886480:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd53163 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5e7 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:0.0:1769074764.886495:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.886498:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074764.886506:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074764.886506:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62688:x1855007616755968:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 295us (1053us total) trans 8589943894 rc 0/0 00000100:00100000:1.0:1769074764.886514:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4641 00010000:00010000:0.0:1769074764.886517:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074764.886560:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074764.886689:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625101056:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074764.886759:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074764.900992:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.901852:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616756096 00000100:00100000:2.0:1769074764.901868:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616756096 00002000:00100000:2.0:1769074764.901961:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3133440->3137535) req@ffff89b546c69880 x1855007616756096/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.901974:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69880 with x1855007616756096 ext(3133440->3137535) 00002000:00010000:2.0:1769074764.901984:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c69880 00000100:00100000:2.0:1769074764.902036:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8393 00000100:00100000:2.0:1769074764.902049:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616756096:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074764.904282:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616756096:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 2211us (3277us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074764.904305:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8393 00002000:00100000:2.0:1769074764.904320:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3133440->3137535) req@ffff89b546c69880 x1855007616756096/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:300/0 lens 488/4536 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074764.904333:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69880 with x1855007616756096 ext(3133440->3137535) 00002000:00010000:2.0:1769074764.904342:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546c69880 00000100:00100000:0.0:1769074764.906463:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625101056:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074764.906494:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074764.906580:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616755712:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 20194us (21232us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074764.906592:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6893 00000100:00100000:3.0:1769074764.912270:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.913194:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074764.913561:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616756352 00000100:00100000:2.0:1769074764.913582:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616756352 00000100:00100000:2.0:1769074764.913655:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4642 00000100:00100000:2.0:1769074764.913666:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62687:x1855007616756352:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074764.914065:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62687:x1855007616756352:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 399us (1824us total) trans 8589943895 rc 0/0 00000100:00100000:2.0:1769074764.914073:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4642 00000100:00100000:3.0:1769074764.915272:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616756608 00000100:00100000:3.0:1769074764.915284:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616756608 00000100:00100000:3.0:1769074764.915344:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13291 00000100:00100000:3.0:1769074764.915354:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62693:x1855007616756608:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074764.915375:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074764.915398:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53243 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b673 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.915479:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915499:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915510:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915525:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915532:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915538:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915663:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915673:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915681:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074764.915702:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074764.915778:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915785:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915791:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915797:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915845:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.915851:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915859:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915872:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074764.915876:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074764.915880:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915886:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5324a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915903:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915909:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915923:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915929:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910d800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074764.915932:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915937:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915944:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ead00) 00010000:00010000:3.0:1769074764.915947:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074764.915950:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074764.915956:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53251 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074764.916253:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53243 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.916272:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074764.916279:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074764.916282:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074764.916285:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:2.0:1769074764.916375:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62693:x1855007616756608:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1022us (3186us total) trans 8589943896 rc 0/0 00000100:00100000:2.0:1769074764.916388:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13291 00000100:00100000:1.0:1769074764.935900:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.935951:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.936116:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616756736 00000100:00100000:3.0:1769074764.936140:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616756736 00002000:00100000:3.0:1769074764.936241:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441e300 x1855007616756736/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.936262:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441e300 with x1855007616756736 ext(1048576->2097151) 00002000:00010000:3.0:1769074764.936271:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441e300 00000100:00100000:3.0:1769074764.936301:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616756864 00000100:00100000:3.0:1769074764.936305:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616756864 00002000:00100000:3.0:1769074764.936320:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54b427b80 x1855007616756864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/0 e 0 to 0 dl 1769074775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074764.936329:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427b80 with x1855007616756864 ext(1048576->2097151) 00002000:00010000:3.0:1769074764.936332:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b427b80 00000100:00100000:3.0:1769074764.937948:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8394 00000100:00100000:3.0:1769074764.937970:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616756736:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074764.940539:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8395 00000100:00100000:1.0:1769074764.940582:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616756864:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074764.958576:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074764.958978:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757120 00000100:00100000:1.0:1769074764.958991:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757120 00000100:00100000:1.0:1769074764.959067:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4643 00000100:00100000:1.0:1769074764.959078:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c7450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62693:x1855007616757120:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074764.959362:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c7450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62693:x1855007616757120:12345-192.168.204.60@tcp:35:cat.0 Request processed in 285us (801us total) trans 8589943897 rc 0/0 00000100:00100000:1.0:1769074764.959371:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4643 00000100:00100000:1.0:1769074764.975024:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616756736:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37050us (39121us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074764.975040:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8394 00002000:00100000:1.0:1769074764.975055:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54441e300 x1855007616756736/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.975066:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441e300 with x1855007616756736 ext(1048576->2097151) 00002000:00010000:1.0:1769074764.975072:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441e300 00000100:00100000:2.0:1769074764.976193:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074764.976333:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757248 00000100:00100000:3.0:1769074764.976348:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757248 00000100:00100000:3.0:1769074764.978559:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13882 00000100:00100000:3.0:1769074764.978590:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616757248:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074764.978617:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074764.978645:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b67a expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074764.978679:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074764.978682:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074764.978692:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074764.978711:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074764.978728:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:3.0:1769074764.978827:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616757248:12345-192.168.204.60@tcp:101:cat.0 Request processed in 252us (2639us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074764.978838:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13882 00000100:00100000:1.0:1769074764.986306:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616756864:12345-192.168.204.60@tcp:3:cat.0 Request processed in 45735us (50339us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074764.986324:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8395 00002000:00100000:1.0:1769074764.986340:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54b427b80 x1855007616756864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:300/0 lens 488/440 e 0 to 0 dl 1769074775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074764.986350:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427b80 with x1855007616756864 ext(1048576->2097151) 00002000:00010000:1.0:1769074764.986356:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b427b80 00000100:00100000:2.0:1769074765.006246:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.006330:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.006466:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757376 00000100:00100000:1.0:1769074765.006476:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757376 00002000:00100000:1.0:1769074765.006562:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b546f95180 x1855007616757376/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074765.006574:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95180 with x1855007616757376 ext(0->1048575) 00002000:00010000:1.0:1769074765.006581:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f95180 00000100:00100000:1.0:1769074765.006622:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757504 00000100:00100000:1.0:1769074765.006628:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757504 00002000:00100000:1.0:1769074765.006654:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546f97b80 x1855007616757504/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074765.006663:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97b80 with x1855007616757504 ext(1048576->2097151) 00002000:00010000:1.0:1769074765.006668:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f97b80 00000100:00100000:1.0:1769074765.006691:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8396 00000100:00100000:1.0:1769074765.006705:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616757376:12345-192.168.204.60@tcp:3:cat.0 00000100:00080000:0.0:1769074765.008142:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074765.008151:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074765.008181:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8397 00000100:00100000:0.0:1769074765.008199:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616757504:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074765.028097:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.028634:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757632 00000100:00100000:1.0:1769074765.028645:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757632 00000100:00100000:1.0:1769074765.028763:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13883 00000100:00100000:1.0:1769074765.028773:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62683:x1855007616757632:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074765.028795:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.028853:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b681 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.028889:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.028898:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074765.028900:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074765.028904:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.028939:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.028954:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 18 pid: 27844 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074765.028968:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625101184:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074765.029060:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.034539:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616757376:12345-192.168.204.60@tcp:3:cat.0 Request processed in 27827us (28294us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074765.034559:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8396 00002000:00100000:1.0:1769074765.034572:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b546f95180 x1855007616757376/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074765.034583:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95180 with x1855007616757376 ext(0->1048575) 00002000:00010000:1.0:1769074765.034589:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f95180 00000100:00100000:1.0:1769074765.036129:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616757504:12345-192.168.204.60@tcp:3:cat.0 Request processed in 27928us (29790us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074765.036143:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8397 00002000:00100000:1.0:1769074765.036154:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546f97b80 x1855007616757504/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074765.036164:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97b80 with x1855007616757504 ext(1048576->2097151) 00002000:00010000:1.0:1769074765.036169:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f97b80 00000100:00080000:0.0:1769074765.037903:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074765.042232:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.042347:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757760 00000100:00100000:3.0:1769074765.042360:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757760 00000100:00100000:3.0:1769074765.042419:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13884 00000100:00100000:3.0:1769074765.042430:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:62685:x1855007616757760:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074765.042480:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074765.042503:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b688 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.042542:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.042550:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.042563:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.042565:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.042570:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1835008->2097151] (req 1835008->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.042607:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.042620:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 19 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:3.0:1769074765.042633:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625101312:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.042737:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.042747:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074765.042757:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625101440:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074765.042793:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074765.046643:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.046765:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625101184:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074765.046798:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.046853:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:1.0:1769074765.046903:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62683:x1855007616757632:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18130us (18817us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.046913:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13883 00000100:00100000:0.0:1769074765.050268:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616757888 00000100:00100000:0.0:1769074765.050290:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616757888 00002000:00100000:0.0:1769074765.050375:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2666496->3137535) req@ffff89b546f96300 x1855007616757888/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.050387:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96300 with x1855007616757888 ext(2666496->3137535) 00010000:00010000:0.0:1769074765.050398:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 17 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:0.0:1769074765.050416:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 17 pid: 27844 timeout: 1852 lvb_type: 0 00002000:00010000:0.0:1769074765.050430:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f96300 00000100:00100000:0.0:1769074765.050486:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 714 00000100:00100000:0.0:1769074765.050496:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616757888:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074765.059434:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.059518:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758016 00000100:00100000:2.0:1769074765.059529:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758016 00000100:00100000:1.0:1769074765.059646:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13292 00000100:00100000:1.0:1769074765.059660:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62695:x1855007616758016:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074765.059696:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.059718:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd5327b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b68f expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.059792:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059840:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059851:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059868:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059874:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059880:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059973:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059984:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.059990:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074765.060034:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074765.060038:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074765.060059:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060069:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060079:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060085:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060091:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060098:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060103:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074765.060107:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060115:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060126:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074765.060130:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074765.060135:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060140:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd53282 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060164:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060171:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060176:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060182:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074765.060185:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060191:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060198:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074765.060200:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074765.060204:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060209:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd53289 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.060238:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd5327b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.060252:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 16 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.060259:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.060261:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074765.060264:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866800, rc 0) 00000100:00100000:1.0:1769074765.060337:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:62695:x1855007616758016:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 682us (916us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.060345:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13292 00000100:00100000:3.0:1769074765.062623:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625101440:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074765.068743:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.068774:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.068831:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.068869:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758400 00000100:00100000:1.0:1769074765.068879:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758400 00000100:00100000:1.0:1769074765.068949:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13885 00000100:00100000:1.0:1769074765.068960:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62693:x1855007616758400:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074765.068980:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.069003:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b696 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.069035:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.069042:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1835008->2097151] (req 1835008->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.069049:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074765.069054:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.069099:0:8398:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1835008->2097151] (req 1835008->2097151) gid 0 flags: 0x60000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.069113:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.069129:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:1.0:1769074765.069205:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62693:x1855007616758400:12345-192.168.204.60@tcp:101:cat.0 Request processed in 245us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.069213:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13885 00000100:00100000:2.0:1769074765.070095:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625101312:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.070142:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1835008->2097151] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.070155:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:2.0:1769074765.070207:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62685:x1855007616757760:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 27779us (27987us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.070214:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13884 00000100:00100000:3.0:1769074765.072651:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758144 00000100:00100000:3.0:1769074765.072671:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758144 00010000:00010000:3.0:1769074765.072731:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074765.072988:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.073019:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758528 00000100:00100000:2.0:1769074765.073023:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758528 00000100:00100000:2.0:1769074765.073060:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13886 00000100:00100000:2.0:1769074765.073070:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616758528:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074765.073088:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.073110:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5329e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b69d expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.073135:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.073148:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5329e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b69d expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.073183:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5329e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b69d expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.073196:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3c00, rc 0) 00000100:00100000:2.0:1769074765.073269:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616758528:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 199us (281us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074765.073277:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13886 00000100:00100000:2.0:1769074765.073320:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.073340:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758656 00000100:00100000:2.0:1769074765.073344:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758656 00000100:00100000:2.0:1769074765.073369:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13887 00000100:00100000:2.0:1769074765.073375:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616758656:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074765.073384:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.073395:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3400/0x7fe2323adfd532a5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6a4 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074765.073455:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625101696:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074765.073514:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074765.073701:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758272 00000100:00100000:3.0:1769074765.073706:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758272 00010000:00010000:3.0:1769074765.073727:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 21 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074765.079386:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616757888:12345-192.168.204.60@tcp:4:cat.0 Request processed in 28876us (32738us total) trans 17179874492 rc 0/0 00000100:00100000:2.0:1769074765.079403:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 714 00002000:00100000:2.0:1769074765.079418:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2666496->3137535) req@ffff89b546f96300 x1855007616757888/t17179874492(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/448 e 0 to 0 dl 1769074776 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.079450:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96300 with x1855007616757888 ext(2666496->3137535) 00010000:00010000:2.0:1769074765.079461:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 19 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.079474:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 19 pid: 27844 timeout: 1852 lvb_type: 0 00002000:00010000:2.0:1769074765.079482:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f96300 00000100:00100000:0.0:1769074765.080329:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6894 00000100:00100000:1.0:1769074765.080426:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616758144:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.080455:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.080482:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.080496:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.080520:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd5322e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b665 expref: 31 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.080539:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.080606:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616758144:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 264us (11877us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.080614:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6894 00000100:00100000:1.0:1769074765.080636:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6895 00000100:00100000:1.0:1769074765.080644:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616758272:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.080655:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.080665:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 21 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.080677:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 21 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.081128:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53266 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b67a expref: 21 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.081145:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.081147:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.081167:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.081178:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.081226:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 29 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.081239:0:33102:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 29 pid: 27844 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074765.081255:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625101824:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074765.081333:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00002000:00010000:2.0:1769074765.084522:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074761 -> 1769074764 00002000:00010000:2.0:1769074765.084534:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074759 -> 1769074764 00002000:00010000:2.0:1769074765.084537:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074761 -> 1769074764 00000100:00100000:2.0:1769074765.084580:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625101696:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074765.085288:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3400/0x7fe2323adfd532a5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6a4 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.085322:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b64eff3400/0x7fe2323adfd532a5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6a4 expref: 12 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.085338:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3400, rc 0) 00000100:00100000:2.0:1769074765.085487:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616758656:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 12110us (12165us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074765.085497:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13887 00000100:00100000:0.0:1769074765.087495:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.087896:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758784 00000100:00100000:3.0:1769074765.087908:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758784 00010000:00010000:2.0:1769074765.088481:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 19 pid: 27844 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074765.088536:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6896 00000100:00100000:2.0:1769074765.088547:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616758784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074765.088577:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074765.088618:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 19 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.088635:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 19 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.088664:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53220 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2666496->3137535) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b657 expref: 19 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.089413:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.089418:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.089428:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.089452:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.089500:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074765.089516:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625101952:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074765.090115:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.092123:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625101824:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074765.092149:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.092897:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616758272:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12248us (24116us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.094037:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6895 00000100:00100000:2.0:1769074765.094552:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.094755:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625101952:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074765.094782:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074765.094866:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616758784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6320us (8279us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.094876:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6896 00000100:00100000:3.0:1769074765.096107:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616758912 00000100:00100000:3.0:1769074765.096120:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616758912 00002000:00100000:3.0:1769074765.096193:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1835008->1839103) req@ffff89b546f96a00 x1855007616758912/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.096204:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96a00 with x1855007616758912 ext(1835008->1839103) 00010000:00010000:3.0:1769074765.096214:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 30 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.096230:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 30 pid: 27844 timeout: 1852 lvb_type: 0 00002000:00010000:3.0:1769074765.096238:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f96a00 00000100:00100000:3.0:1769074765.096272:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 715 00000100:00100000:3.0:1769074765.096283:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2407:x1855007616758912:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074765.096490:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2407:x1855007616758912:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 208us (1941us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074765.096498:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 715 00002000:00100000:3.0:1769074765.096504:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1835008->1839103) req@ffff89b546f96a00 x1855007616758912/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/4536 e 0 to 0 dl 1769074776 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.096512:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96a00 with x1855007616758912 ext(1835008->1839103) 00010000:00010000:3.0:1769074765.096518:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 30 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.096528:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 30 pid: 27844 timeout: 1852 lvb_type: 0 00002000:00010000:3.0:1769074765.096535:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f96a00 00000100:00100000:2.0:1769074765.111267:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.111388:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616759680 00000100:00100000:3.0:1769074765.111400:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616759680 00002000:00100000:3.0:1769074765.111463:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1835008->2097151) req@ffff89b546f95880 x1855007616759680/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.111474:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95880 with x1855007616759680 ext(1835008->2097151) 00010000:00010000:3.0:1769074765.111483:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 30 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.111499:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 30 pid: 27844 timeout: 1852 lvb_type: 0 00002000:00010000:3.0:1769074765.111507:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f95880 00000100:00100000:3.0:1769074765.111547:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 716 00000100:00100000:3.0:1769074765.111559:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616759680:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074765.113044:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.113138:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.113162:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616759552 00000100:00100000:0.0:1769074765.113169:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616759552 00002000:00100000:0.0:1769074765.113210:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54759c700 x1855007616759552/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.113222:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759c700 with x1855007616759552 ext(2097152->4194303) 00002000:00010000:0.0:1769074765.113229:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759c700 00000100:00100000:0.0:1769074765.113260:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8398 00000100:00100000:0.0:1769074765.113270:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616759552:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074765.121563:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616759808 00000100:00100000:2.0:1769074765.121576:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616759808 00000100:00100000:2.0:1769074765.121635:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13888 00000100:00100000:2.0:1769074765.121644:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62685:x1855007616759808:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074765.121660:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.121678:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6b9 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.121707:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.121718:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.121726:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 33 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.121732:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.121734:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.121738:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1048576->1658879] (req 1048576->1658879) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.121769:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 33 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.121779:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 33 pid: 9048 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074765.121793:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625102080:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.121896:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.121906:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 9 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074765.121927:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f96680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625102208:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.121952:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.121961:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 19 pid: 9048 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074765.121969:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625102336:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074765.121984:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.124165:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.124231:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.124280:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616759168 00000100:00100000:1.0:1769074765.124292:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616759168 00000100:00100000:1.0:1769074765.124352:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616759424 00000100:00100000:1.0:1769074765.124356:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616759424 00000100:00100000:1.0:1769074765.124397:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13293 00000100:00100000:1.0:1769074765.124409:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:62695:x1855007616759168:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074765.124446:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.124476:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd532b3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6ab expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.124639:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.124660:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.124671:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.124682:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 5/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.124724:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.124738:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 17 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074765.124752:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625102464:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074765.124888:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.124901:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 24 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074765.124911:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625102592:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074765.124936:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00080000:1.0:1769074765.125087:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074765.126137:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13294 00000100:00100000:2.0:1769074765.126173:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62697:x1855007616759424:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074765.126200:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.126228:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532c1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6b2 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.126345:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074765.126360:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074765.126368:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074765.134209:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2408:x1855007616759680:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 22634us (22938us total) trans 21474841175 rc 0/0 00000100:00100000:3.0:1769074765.134230:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 716 00002000:00100000:3.0:1769074765.134245:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1835008->2097151) req@ffff89b546f95880 x1855007616759680/t21474841175(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/448 e 0 to 0 dl 1769074776 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.134257:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95880 with x1855007616759680 ext(1835008->2097151) 00010000:00010000:3.0:1769074765.134269:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 34 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.134286:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 34 pid: 27844 timeout: 1852 lvb_type: 0 00002000:00010000:3.0:1769074765.134294:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546f95880 00000100:00100000:1.0:1769074765.143123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.143156:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.143209:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.143233:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.143262:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760064 00000100:00100000:1.0:1769074765.143271:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760064 00010000:00010000:1.0:1769074765.143313:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 10 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:1.0:1769074765.143356:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616759936 00000100:00100000:1.0:1769074765.143360:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616759936 00010000:00010000:1.0:1769074765.143372:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 34 pid: 9048 timeout: 1852 lvb_type: 1 00000100:00100000:1.0:1769074765.143394:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760320 00000100:00100000:1.0:1769074765.143397:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760320 00010000:00010000:1.0:1769074765.143411:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 26 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074765.143424:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760192 00000100:00100000:1.0:1769074765.143427:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760192 00010000:00010000:1.0:1769074765.143450:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 18 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:1.0:1769074765.143465:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6897 00000100:00100000:1.0:1769074765.143473:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616760064:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.143489:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.143519:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 10 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.143534:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 10 pid: 27844 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.143579:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.143845:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616760064:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 334us (697us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.143855:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6897 00000100:00100000:1.0:1769074765.143880:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6898 00000100:00100000:1.0:1769074765.143887:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616759936:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.143895:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.143903:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 34 pid: 9048 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.144023:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 34 pid: 9048 timeout: 1852 lvb_type: 1 00010000:00010000:1.0:1769074765.144038:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.144061:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616759936:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 174us (903us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.144068:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6898 00000100:00100000:1.0:1769074765.144083:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6899 00000100:00100000:1.0:1769074765.144090:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2408:x1855007616760320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.144099:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.144104:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 26 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.144113:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 26 pid: 6514 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.144126:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.144131:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.144141:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.144146:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.144165:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2408:x1855007616760320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 76us (956us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.144172:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6899 00000100:00100000:1.0:1769074765.144184:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6900 00000100:00100000:1.0:1769074765.144190:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2408:x1855007616760192:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.144197:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.144201:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 18 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.144215:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 18 pid: 6512 timeout: 1852 lvb_type: 0 00010000:00010000:1.0:1769074765.144226:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.144230:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.144239:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.144246:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.144253:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074765.144259:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074765.144264:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074765.144291:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.144310:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2408:x1855007616760192:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 120us (1076us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.144317:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6900 00000100:00100000:0.0:1769074765.145315:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616759552:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 32035us (32165us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074765.145333:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8398 00002000:00100000:0.0:1769074765.145345:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54759c700 x1855007616759552/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.145356:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759c700 with x1855007616759552 ext(2097152->4194303) 00002000:00010000:0.0:1769074765.145362:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759c700 00010000:00010000:2.0:1769074765.146043:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5317f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b60a expref: 33 pid: 9048 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074765.146063:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625102080:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.146092:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074765.146099:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074765.146125:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074765.146310:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532c1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 27 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.146328:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.146339:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074765.146342:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1769074765.146350:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4a00, rc 0) 00000100:00100000:2.0:1769074765.146470:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+26:62697:x1855007616759424:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20320us (22238us total) trans 8589943898 rc 0/0 00000100:00100000:2.0:1769074765.146478:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13294 00010000:00010000:2.0:1769074765.146526:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd531fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b649 expref: 9 pid: 27844 timeout: 1852 lvb_type: 1 00000100:00100000:2.0:1769074765.146536:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f96680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625102208:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074765.146548:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625102336:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.146742:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->1658879] (req 1048576->1658879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 32 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.146760:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4000, rc 0) 00000100:00100000:2.0:1769074765.147203:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62685:x1855007616759808:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 25559us (34165us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.147211:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13888 00010000:00010000:3.0:1769074765.147608:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd53290 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b68f expref: 17 pid: 6512 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074765.147627:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625102464:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.147642:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53258 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b673 expref: 25 pid: 6514 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074765.147653:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625102592:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.147674:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.147683:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.147689:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074765.147711:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074765.147887:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd532b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.147901:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.148368:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074765.148373:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1769074765.148377:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866a00, rc 0) 00000100:00100000:3.0:1769074765.148443:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62695:x1855007616759168:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 24026us (24294us total) trans 8589943899 rc 0/0 00000100:00100000:3.0:1769074765.148452:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13293 00000100:00100000:2.0:1769074765.148651:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.148755:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.148779:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760448 00000100:00100000:3.0:1769074765.148957:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760576 00000100:00100000:3.0:1769074765.148967:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760576 00010000:00010000:3.0:1769074765.149002:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 32 pid: 27844 timeout: 1852 lvb_type: 0 00000100:00100000:3.0:1769074765.149038:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6901 00000100:00100000:3.0:1769074765.149048:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616760576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074765.149063:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1769074765.149082:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074763 -> 1769074764 00002000:00010000:3.0:1769074765.149086:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074763 -> 1769074764 00010000:00010000:3.0:1769074765.149093:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 32 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.149108:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 32 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.149129:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53274 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1835008->2097151) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b688 expref: 32 pid: 27844 timeout: 1852 lvb_type: 0 00010000:00010000:3.0:1769074765.149143:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.149145:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.149152:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.149162:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.149201:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074765.149215:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625102720:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074765.149274:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074765.150091:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760448 00010000:00010000:2.0:1769074765.150119:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00000100:00100000:2.0:1769074765.150156:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6902 00000100:00100000:2.0:1769074765.150163:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616760448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074765.150171:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074765.150186:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.150197:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.150215:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd53171 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5f5 expref: 20 pid: 9048 timeout: 1852 lvb_type: 0 00010000:00010000:2.0:1769074765.150226:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.150228:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.150233:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.150241:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.150278:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074765.150288:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625102848:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074765.150341:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074765.156530:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625102720:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074765.156563:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074765.156707:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616760576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7659us (7950us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074765.156718:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6901 00000100:00100000:1.0:1769074765.164442:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.166703:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760704 00000100:00100000:3.0:1769074765.166720:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760704 00000100:00100000:2.0:1769074765.167116:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625102848:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074765.167147:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074765.167265:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616760448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17098us (18613us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.167277:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6902 00000100:00100000:3.0:1769074765.167416:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13295 00000100:00100000:3.0:1769074765.167431:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62695:x1855007616760704:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074765.167475:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074765.167508:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd532dd lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6c0 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.167664:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.167688:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.167699:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.167707:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 5/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.167753:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.167771:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 24 pid: 6514 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.167837:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625102976:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.167950:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.167961:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 18 pid: 6512 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.167974:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625103104:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074765.168008:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.169453:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.169547:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760832 00000100:00100000:2.0:1769074765.169555:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760832 00000100:00100000:2.0:1769074765.169687:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13889 00000100:00100000:2.0:1769074765.169697:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616760832:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074765.169716:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.169737:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6c7 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.169759:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.169762:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.169771:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 32 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.169804:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 32 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.169855:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4e00, rc 0) 00000100:00100000:2.0:1769074765.169908:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616760832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 212us (474us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.170081:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13889 00000100:00100000:1.0:1769074765.175012:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.175112:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616760960 00000100:00100000:0.0:1769074765.175122:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616760960 00002000:00100000:0.0:1769074765.175182:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54b424700 x1855007616760960/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.175194:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b424700 with x1855007616760960 ext(1048576->2097151) 00002000:00010000:0.0:1769074765.175201:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b424700 00000100:00100000:0.0:1769074765.175233:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8399 00000100:00100000:0.0:1769074765.175243:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616760960:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074765.198355:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.198402:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.198429:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.198535:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761088 00000100:00100000:0.0:1769074765.198545:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761088 00002000:00100000:0.0:1769074765.198604:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1654784->1658879) req@ffff89b54b425500 x1855007616761088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.198615:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425500 with x1855007616761088 ext(1654784->1658879) 00002000:00010000:0.0:1769074765.198621:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425500 00000100:00100000:0.0:1769074765.198658:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8400 00000100:00100000:0.0:1769074765.198668:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616761088:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074765.198856:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616761088:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 188us (515us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074765.198867:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8400 00002000:00100000:0.0:1769074765.198874:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1654784->1658879) req@ffff89b54b425500 x1855007616761088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/4536 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.198881:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425500 with x1855007616761088 ext(1654784->1658879) 00002000:00010000:0.0:1769074765.198883:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425500 00000100:00100000:2.0:1769074765.199254:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761216 00000100:00100000:2.0:1769074765.199260:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761216 00010000:00010000:2.0:1769074765.199289:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 25 pid: 6514 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.199327:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761344 00000100:00100000:2.0:1769074765.199332:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761344 00010000:00010000:2.0:1769074765.199345:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 19 pid: 6512 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.199365:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6903 00000100:00100000:2.0:1769074765.199372:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2406:x1855007616761216:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074765.199386:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074765.199392:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 25 pid: 6514 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.199406:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 25 pid: 6514 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.199442:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074765.199447:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074765.199458:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074765.199464:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074765.199471:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074765.199508:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2406:x1855007616761216:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 135us (1105us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.199515:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6903 00000100:00100000:2.0:1769074765.199531:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6904 00000100:00100000:2.0:1769074765.199537:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2406:x1855007616761344:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074765.199546:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074765.199550:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 5/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 19 pid: 6512 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.199560:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 19 pid: 6512 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.199573:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074765.199577:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074765.199584:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074765.199603:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074765.199620:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074765.199639:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2406:x1855007616761344:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 102us (1209us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.199645:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6904 00010000:00010000:3.0:1769074765.200244:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd532c8 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b2 expref: 24 pid: 6514 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.200284:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625102976:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.200303:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd532ba lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ab expref: 18 pid: 6512 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.200315:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625103104:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.200349:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.200358:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.200362:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074765.200444:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:3.0:1769074765.200450:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074765.200473:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074765.200505:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd532dd lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.200518:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.200527:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074765.200530:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074765.200533:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:3.0:1769074765.200907:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62695:x1855007616760704:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 33478us (36487us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074765.200917:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13295 00000100:00100000:0.0:1769074765.206286:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.206495:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761472 00000100:00100000:1.0:1769074765.206506:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761472 00002000:00100000:1.0:1769074765.206554:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54759c000 x1855007616761472/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074765.206638:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759c000 with x1855007616761472 ext(4194304->4808703) 00002000:00010000:1.0:1769074765.206645:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759c000 00000100:00100000:1.0:1769074765.206721:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8401 00000100:00100000:1.0:1769074765.206734:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616761472:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:0.0:1769074765.215940:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074765.218425:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616760960:12345-192.168.204.60@tcp:3:cat.0 Request processed in 43167us (43408us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074765.218450:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8399 00002000:00100000:0.0:1769074765.218466:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54b424700 x1855007616760960/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074765.218475:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b424700 with x1855007616760960 ext(1048576->2097151) 00002000:00010000:0.0:1769074765.218482:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b424700 00000100:00100000:2.0:1769074765.220127:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616761472:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13390us (13845us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074765.220138:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8401 00002000:00100000:2.0:1769074765.220147:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54759c000 x1855007616761472/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.220167:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759c000 with x1855007616761472 ext(4194304->4808703) 00002000:00010000:2.0:1769074765.220172:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759c000 00000100:00100000:3.0:1769074765.221105:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.221219:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.221294:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616762112 00000100:00100000:3.0:1769074765.221302:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616762112 00000100:00100000:3.0:1769074765.221352:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4644 00000100:00100000:3.0:1769074765.221402:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:62685:x1855007616762112:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074765.221429:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761728 00000100:00100000:2.0:1769074765.221444:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761728 00000100:00100000:2.0:1769074765.221480:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13890 00000100:00100000:2.0:1769074765.221486:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616761728:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074765.221495:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.221512:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd532f2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6d5 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074765.221572:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103232:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074765.221609:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62685:x1855007616762112:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 249us (390us total) trans 8589943900 rc 0/0 00000100:00100000:3.0:1769074765.221617:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4644 00000100:00100000:2.0:1769074765.221628:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074765.224762:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.224916:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:3.0:1769074765.232897:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074763 -> 1769074764 00002000:00010000:3.0:1769074765.232908:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074763 -> 1769074764 00002000:00010000:3.0:1769074765.232911:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074763 -> 1769074764 00000100:00100000:3.0:1769074765.232934:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f95c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103232:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074765.232970:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd532f2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6d5 expref: 10 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.232991:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd532f2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6d5 expref: 10 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.233005:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea5400, rc 0) 00000100:00100000:3.0:1769074765.233075:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616761728:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11588us (11972us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074765.233084:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13890 00000100:00100000:0.0:1769074765.233242:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761600 00000100:00100000:2.0:1769074765.233525:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.234348:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616761856 00000100:00100000:1.0:1769074765.234356:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761856 00000100:00100000:1.0:1769074765.234425:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616762240 00000100:00100000:1.0:1769074765.234430:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616762240 00000100:00100000:1.0:1769074765.234473:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13891 00000100:00100000:1.0:1769074765.234483:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616761856:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074765.234499:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.234516:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6dc expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.234538:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.234541:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.234551:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 12 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074765.234567:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 12 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074765.234579:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:1.0:1769074765.234626:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616761856:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 144us (9710us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.234633:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13891 00000100:00100000:1.0:1769074765.234651:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13892 00000100:00100000:1.0:1769074765.234658:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62693:x1855007616762240:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074765.234666:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.234675:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6e3 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.234694:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.234701:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074765.234703:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074765.234706:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 20 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.234735:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 31 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.234748:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 31 pid: 27844 timeout: 1853 lvb_type: 0 00000100:00100000:1.0:1769074765.234764:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625103360:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074765.234836:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b647316240 going to sleep for 11 seconds 00000100:00100000:0.0:1769074765.235714:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616761600 00000100:00100000:2.0:1769074765.236098:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13893 00000100:00100000:2.0:1769074765.236109:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616761600:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074765.236126:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.236145:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6ce expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.236167:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.236169:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.236178:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.236193:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.236205:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3400, rc 0) 00000100:00100000:2.0:1769074765.236273:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616761600:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 165us (11525us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.236280:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13893 00000100:00100000:2.0:1769074765.238683:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.238761:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616762368 00000100:00100000:3.0:1769074765.238772:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616762368 00000100:00100000:3.0:1769074765.238858:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13894 00000100:00100000:3.0:1769074765.238869:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62697:x1855007616762368:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074765.238890:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074765.238912:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6ea expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.238944:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.238954:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.238962:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 31 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.238969:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.238971:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.238976:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [1601536->1998847] (req 1601536->1998847) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.239025:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 31 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.239039:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 31 pid: 27844 timeout: 1853 lvb_type: 1 00000100:00100000:3.0:1769074765.239053:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103488:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.239154:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 9 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.239163:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 9 pid: 27844 timeout: 1853 lvb_type: 1 00000100:00100000:3.0:1769074765.239171:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103616:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074765.239202:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.239210:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.239220:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103744:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074765.239241:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074765.243062:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.243186:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625103360:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074765.243233:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.243260:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052000, rc 0) 00000100:00100000:1.0:1769074765.243338:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62693:x1855007616762240:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8678us (9812us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.243349:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13892 00000100:00100000:3.0:1769074765.243602:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616762496 00000100:00100000:3.0:1769074765.243610:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616762496 00002000:00100000:3.0:1769074765.243685:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1658879) req@ffff89b54d0e4e00 x1855007616762496/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.243695:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4e00 with x1855007616762496 ext(1048576->1658879) 00010000:00010000:3.0:1769074765.243702:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 32 pid: 27844 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.243715:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 32 pid: 27844 timeout: 1853 lvb_type: 0 00002000:00010000:3.0:1769074765.243722:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d0e4e00 00000100:00100000:3.0:1769074765.243751:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 717 00000100:00100000:3.0:1769074765.243760:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616762496:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074765.246126:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.246237:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103488:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074765.246254:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103616:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074765.246317:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616762624 00000100:00100000:0.0:1769074765.246324:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616762624 00010000:00010000:0.0:1769074765.246374:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 34 pid: 27844 timeout: 1853 lvb_type: 1 00000100:00100000:0.0:1769074765.246419:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6905 00000100:00100000:0.0:1769074765.246429:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2408:x1855007616762624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074765.246458:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074765.246485:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 34 pid: 27844 timeout: 1853 lvb_type: 1 00010000:00010000:0.0:1769074765.246498:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 34 pid: 27844 timeout: 1853 lvb_type: 1 00010000:00010000:0.0:1769074765.246521:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd532eb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c7 expref: 34 pid: 27844 timeout: 1853 lvb_type: 1 00010000:00010000:0.0:1769074765.246540:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074765.246608:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616762624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 180us (501us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.246616:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6905 00000100:00100000:2.0:1769074765.249992:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.250170:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625103744:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074765.250213:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1601536->1998847] (req 1601536->1998847) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.250235:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b4a00, rc 0) 00000100:00100000:1.0:1769074765.250308:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62697:x1855007616762368:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11440us (11634us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.250317:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13894 00000100:00100000:0.0:1769074765.251250:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616762752 00000100:00100000:0.0:1769074765.251262:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616762752 00010000:00010000:0.0:1769074765.251302:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 10 pid: 27844 timeout: 1853 lvb_type: 1 00000100:00100000:0.0:1769074765.251342:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6906 00000100:00100000:0.0:1769074765.251349:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616762752:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074765.251362:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074765.251380:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 10 pid: 27844 timeout: 1853 lvb_type: 1 00010000:00010000:0.0:1769074765.251389:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 10 pid: 27844 timeout: 1853 lvb_type: 1 00010000:00010000:0.0:1769074765.251411:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53307 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ce expref: 10 pid: 27844 timeout: 1853 lvb_type: 1 00010000:00010000:0.0:1769074765.251424:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074765.251483:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616762752:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 134us (1509us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.251491:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6906 00000100:00100000:1.0:1769074765.257916:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.258107:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.258141:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763264 00000100:00100000:1.0:1769074765.258153:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763264 00000100:00100000:1.0:1769074765.258215:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2298 00000100:00100000:1.0:1769074765.258224:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616763264:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074765.258305:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616763264:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 82us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.258316:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2298 00000100:00080000:1.0:1769074765.258343:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074765.258424:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.258456:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763008 00000100:00100000:1.0:1769074765.258461:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763008 00000100:00100000:1.0:1769074765.258483:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13296 00000100:00100000:1.0:1769074765.258489:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62695:x1855007616763008:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074765.258501:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.258523:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd53315 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.258628:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258647:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258661:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258676:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258683:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258690:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258697:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b052e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074765.258700:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258709:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 4/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258721:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.258723:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074765.258727:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 2/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258734:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd5331c lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074765.258768:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd53315 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f1 expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.258777:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.258780:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074765.258783:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd53315 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 11 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f1 expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.258790:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:1.0:1769074765.258843:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62695:x1855007616763008:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 353us (419us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074765.258849:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13296 00000100:00100000:1.0:1769074765.258907:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.258923:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763520 00000100:00100000:1.0:1769074765.258926:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763520 00000100:00100000:1.0:1769074765.258951:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2299 00000100:00100000:1.0:1769074765.258957:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616763520:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074765.258982:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616763520:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (75us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.258988:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2299 00000100:00080000:1.0:1769074765.259001:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074765.259075:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.259088:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763392 00000100:00100000:1.0:1769074765.259092:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763392 00000100:00100000:1.0:1769074765.259116:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2300 00000100:00100000:1.0:1769074765.259122:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2407:x1855007616763392:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074765.259145:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2407:x1855007616763392:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 23us (71us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.259151:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2300 00000100:00080000:1.0:1769074765.259163:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074765.259274:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763136 00000100:00100000:0.0:1769074765.259279:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763136 00000100:00100000:0.0:1769074765.259318:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 327 00000100:00100000:0.0:1769074765.259326:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2406:x1855007616763136:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074765.259360:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2406:x1855007616763136:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 36us (1458us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.259366:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 327 00000100:00100000:3.0:1769074765.263364:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616762496:12345-192.168.204.60@tcp:4:cat.0 Request processed in 19592us (20303us total) trans 21474841176 rc 0/0 00000100:00100000:3.0:1769074765.263379:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 717 00002000:00100000:3.0:1769074765.263392:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1658879) req@ffff89b54d0e4e00 x1855007616762496/t21474841176(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/448 e 0 to 0 dl 1769074776 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.263403:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4e00 with x1855007616762496 ext(1048576->1658879) 00010000:00010000:3.0:1769074765.263413:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.263427:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 1853 lvb_type: 0 00002000:00010000:3.0:1769074765.263454:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d0e4e00 00000100:00100000:0.0:1769074765.264452:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.264995:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763648 00000100:00100000:3.0:1769074765.265010:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763648 00010000:00010000:3.0:1769074765.265060:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 21 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.265110:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6907 00000100:00100000:3.0:1769074765.265121:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616763648:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074765.265139:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074765.265165:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 21 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.265178:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 21 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.265207:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53297 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b696 expref: 21 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.265227:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.265229:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.265239:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.265251:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.265302:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074765.265318:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625103872:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074765.265658:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.265733:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.265946:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616763904 00000100:00100000:3.0:1769074765.265953:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616763904 00000100:00100000:0.0:1769074765.267841:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4645 00000100:00100000:0.0:1769074765.267857:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:62695:x1855007616763904:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074765.268474:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:62695:x1855007616763904:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 618us (2738us total) trans 8589943901 rc 0/0 00000100:00100000:2.0:1769074765.268484:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4645 00000100:00100000:1.0:1769074765.276069:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.276253:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764032 00000100:00100000:0.0:1769074765.276262:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764032 00010000:00010000:0.0:1769074765.276304:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.276337:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625103872:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074765.276353:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6908 00000100:00100000:0.0:1769074765.276362:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616764032:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074765.276362:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074765.276381:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074765.276404:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 1853 lvb_type: 0 00010000:00010000:0.0:1769074765.276416:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.276423:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616763648:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11304us (11998us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.276450:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6907 00010000:00010000:0.0:1769074765.276454:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd532ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1658879) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6b9 expref: 33 pid: 27844 timeout: 1853 lvb_type: 0 00010000:00010000:0.0:1769074765.276470:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074765.276472:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074765.276479:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.276487:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.276522:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074765.276539:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625104000:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074765.278262:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.279072:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.279286:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764160 00000100:00100000:3.0:1769074765.279297:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764160 00002000:00100000:2.0:1769074765.279441:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1994752->1998847) req@ffff89b54b425f80 x1855007616764160/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.279453:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425f80 with x1855007616764160 ext(1994752->1998847) 00002000:00010000:2.0:1769074765.279459:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425f80 00000100:00100000:2.0:1769074765.279491:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8402 00000100:00100000:2.0:1769074765.279501:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616764160:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074765.279672:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616764160:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 171us (603us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074765.279680:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8402 00002000:00100000:2.0:1769074765.279687:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1994752->1998847) req@ffff89b54b425f80 x1855007616764160/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/4536 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.279695:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425f80 with x1855007616764160 ext(1994752->1998847) 00002000:00010000:2.0:1769074765.279698:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425f80 00000100:00100000:0.0:1769074765.283111:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.286222:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625104000:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074765.286265:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.286375:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616764032:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10012us (10314us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.286389:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6908 00000100:00100000:2.0:1769074765.286867:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764416 00000100:00100000:2.0:1769074765.286878:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764416 00000100:00100000:2.0:1769074765.286978:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4646 00000100:00100000:2.0:1769074765.286988:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62697:x1855007616764416:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074765.287578:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+25:62697:x1855007616764416:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 588us (4473us total) trans 8589943902 rc 0/0 00000100:00100000:2.0:1769074765.287586:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4646 00000100:00100000:1.0:1769074765.287760:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.287913:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764544 00000100:00100000:2.0:1769074765.287919:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764544 00002000:00100000:2.0:1769074765.287952:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54b425c00 x1855007616764544/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.287962:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425c00 with x1855007616764544 ext(1048576->2097151) 00002000:00010000:2.0:1769074765.287968:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425c00 00000100:00100000:2.0:1769074765.287992:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8403 00000100:00100000:2.0:1769074765.288001:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616764544:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074765.295573:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616764544:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 7561us (7799us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074765.295588:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8403 00002000:00100000:2.0:1769074765.295600:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54b425c00 x1855007616764544/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.295610:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425c00 with x1855007616764544 ext(1048576->2097151) 00002000:00010000:2.0:1769074765.295617:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425c00 00000100:00100000:1.0:1769074765.304099:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.304241:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764672 00000100:00100000:3.0:1769074765.304296:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764672 00000100:00100000:3.0:1769074765.304374:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13895 00000100:00100000:3.0:1769074765.304382:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b35ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62693:x1855007616764672:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074765.304398:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074765.304415:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6f8 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.304457:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.304459:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.304473:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.304491:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.304505:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5200, rc 0) 00000100:00100000:3.0:1769074765.304565:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b35ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62693:x1855007616764672:12345-192.168.204.60@tcp:101:cat.0 Request processed in 183us (476us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074765.304572:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13895 00000100:00100000:1.0:1769074765.314969:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.315049:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764800 00000100:00100000:2.0:1769074765.315058:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764800 00002000:00100000:2.0:1769074765.315099:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54b425880 x1855007616764800/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.315112:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425880 with x1855007616764800 ext(2097152->4194303) 00002000:00010000:2.0:1769074765.315117:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b425880 00000100:00100000:2.0:1769074765.315184:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8404 00000100:00100000:2.0:1769074765.315191:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616764800:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074765.331996:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616764800:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16790us (17020us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074765.332011:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8404 00002000:00100000:2.0:1769074765.332025:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54b425880 x1855007616764800/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.332035:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425880 with x1855007616764800 ext(2097152->4194303) 00002000:00010000:2.0:1769074765.332042:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b425880 00000100:00100000:1.0:1769074765.349351:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.349477:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616764928 00000100:00100000:2.0:1769074765.349489:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616764928 00002000:00100000:2.0:1769074765.349546:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54b427480 x1855007616764928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.349647:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427480 with x1855007616764928 ext(4194304->4808703) 00002000:00010000:2.0:1769074765.349655:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b427480 00000100:00100000:2.0:1769074765.349691:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8405 00000100:00100000:2.0:1769074765.349702:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616764928:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074765.371596:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616764928:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21876us (22238us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074765.371611:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8405 00002000:00100000:2.0:1769074765.371628:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54b427480 x1855007616764928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.371639:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427480 with x1855007616764928 ext(4194304->4808703) 00002000:00010000:2.0:1769074765.371646:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b427480 00000100:00100000:1.0:1769074765.417066:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.417697:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616765184 00000100:00100000:2.0:1769074765.417708:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616765184 00000100:00100000:2.0:1769074765.417785:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13297 00000100:00100000:2.0:1769074765.417794:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62704:x1855007616765184:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074765.417853:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.417873:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5332a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b6ff expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.417944:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.417959:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.417970:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.417983:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.417990:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.417996:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418100:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418112:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418119:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074765.418129:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074765.418219:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418226:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418233:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418240:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418550:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b64eff3400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074765.418565:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418577:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418590:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074765.418593:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074765.418598:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418604:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3400/0x7fe2323adfd53331 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418622:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418627:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418633:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418637:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b64eff3e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074765.418640:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418644:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418651:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4a00) 00010000:00010000:2.0:1769074765.418653:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074765.418655:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418660:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53338 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.418709:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd5332a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 234 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.418723:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 233 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.418730:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074765.418732:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074765.418735:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff2800, rc 0) 00000100:00100000:2.0:1769074765.418856:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62704:x1855007616765184:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1062us (1802us total) trans 8589943903 rc 0/0 00000100:00100000:2.0:1769074765.418864:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13297 00000100:00100000:1.0:1769074765.433792:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.434085:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616765440 00000100:00100000:3.0:1769074765.434095:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616765440 00000100:00100000:3.0:1769074765.434161:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4647 00000100:00100000:3.0:1769074765.434171:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62704:x1855007616765440:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074765.434424:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62704:x1855007616765440:12345-192.168.204.60@tcp:35:cat.0 Request processed in 254us (639us total) trans 8589943904 rc 0/0 00000100:00100000:3.0:1769074765.434440:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4647 00000100:00100000:1.0:1769074765.449035:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.449165:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.451743:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616765696 00000100:00100000:3.0:1769074765.451759:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616765696 00000100:00100000:3.0:1769074765.451837:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616765568 00000100:00100000:3.0:1769074765.451840:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616765568 00000100:00100000:3.0:1769074765.451870:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13896 00000100:00100000:3.0:1769074765.451881:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616765696:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074765.451899:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1769074765.456043:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13897 00000100:00100000:0.0:1769074765.456070:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616765568:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074765.456091:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074765.456111:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5334d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b706 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074765.456185:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625104128:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074765.456276:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00010000:00010000:3.0:1769074765.456828:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b70d expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.456846:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.456848:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.456857:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 35 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.456873:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 35 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.456890:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b4000, rc 0) 00000100:00100000:3.0:1769074765.456926:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2408:x1855007616765696:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5048us (7903us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074765.456933:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13896 00002000:00010000:0.0:1769074765.460402:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074764 -> 1769074765 00000100:00100000:0.0:1769074765.460454:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625104128:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074765.460491:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5334d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b706 expref: 34 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074765.460511:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5334d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b706 expref: 34 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074765.460523:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446200, rc 0) 00000100:00100000:0.0:1769074765.463922:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616765568:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7192us (14083us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074765.463935:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13897 00000100:00100000:1.0:1769074765.483422:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.483539:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616765824 00000100:00100000:0.0:1769074765.483550:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616765824 00000100:00100000:0.0:1769074765.483623:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13898 00000100:00100000:0.0:1769074765.483633:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:62704:x1855007616765824:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074765.483653:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074765.483673:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b714 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.483703:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.483716:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074765.483719:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074765.483723:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 34 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.483762:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.483786:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 19 pid: 9048 timeout: 1853 lvb_type: 0 00000100:00100000:0.0:1769074765.483800:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625104256:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074765.483920:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.491012:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.491113:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616765952 00000100:00100000:2.0:1769074765.491125:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616765952 00002000:00100000:2.0:1769074765.491180:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1601536->1998847) req@ffff89b547ef5c00 x1855007616765952/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.491194:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5c00 with x1855007616765952 ext(1601536->1998847) 00010000:00010000:2.0:1769074765.491204:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 20 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.491220:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 20 pid: 9048 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074765.491228:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef5c00 00000100:00100000:2.0:1769074765.491278:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 718 00000100:00100000:2.0:1769074765.491288:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616765952:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074765.492165:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625104256:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074765.492206:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 34 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.492226:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446200, rc 0) 00000100:00100000:0.0:1769074765.492290:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:62704:x1855007616765824:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8658us (8879us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.492299:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13898 00000100:00100000:2.0:1769074765.501088:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616765952:12345-192.168.204.60@tcp:4:cat.0 Request processed in 9790us (10080us total) trans 21474841177 rc 0/0 00000100:00100000:2.0:1769074765.501104:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 718 00002000:00100000:2.0:1769074765.501117:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1601536->1998847) req@ffff89b547ef5c00 x1855007616765952/t21474841177(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/448 e 0 to 0 dl 1769074776 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.501128:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5c00 with x1855007616765952 ext(1601536->1998847) 00010000:00010000:2.0:1769074765.501139:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.501160:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074765.501168:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b547ef5c00 00000100:00100000:1.0:1769074765.503104:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.503444:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616766208 00000100:00100000:2.0:1769074765.503456:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616766208 00000100:00100000:2.0:1769074765.503523:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13298 00000100:00100000:2.0:1769074765.503533:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62707:x1855007616766208:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074765.503704:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.503729:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd53362 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b71b expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.503968:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.503989:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.503998:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 235 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.504007:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 5/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074765.504045:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 235 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.504062:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 235 pid: 8855 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.504077:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625104384:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.504233:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 17 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.504243:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 17 pid: 6512 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.504262:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625104512:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074765.504299:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.510528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.513224:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616766336 00000100:00100000:2.0:1769074765.513243:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616766336 00010000:00010000:2.0:1769074765.513311:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.513363:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6909 00000100:00100000:2.0:1769074765.513374:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616766336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074765.513391:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074765.513413:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.513425:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.513477:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd5330e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1601536->1998847) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ea expref: 22 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.513494:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.513496:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.513504:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.513514:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.513565:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074765.513579:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625104640:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074765.513682:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:2.0:1769074765.515002:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.515052:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.515138:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625104384:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074765.515151:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625104512:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074765.515179:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000100:00100000:1.0:1769074765.515256:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616766464 00000100:00100000:1.0:1769074765.515264:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616766464 00010000:00010000:1.0:1769074765.515293:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 236 pid: 8855 timeout: 1853 lvb_type: 0 00000100:00100000:1.0:1769074765.515322:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616766592 00000100:00100000:1.0:1769074765.515325:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616766592 00010000:00010000:1.0:1769074765.515338:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 18 pid: 6512 timeout: 1853 lvb_type: 0 00000100:00100000:1.0:1769074765.515366:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6910 00000100:00100000:1.0:1769074765.515374:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:2406:x1855007616766464:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.515386:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.515392:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 236 pid: 8855 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074765.515402:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 236 pid: 8855 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074765.515421:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2800/0x7fe2323adfd5333f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6ff expref: 236 pid: 8855 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074765.515455:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.515459:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074765.515469:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074765.515475:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.515520:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:2406:x1855007616766464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 145us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.515527:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6910 00000100:00100000:1.0:1769074765.515547:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6911 00000100:00100000:1.0:1769074765.515553:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2406:x1855007616766592:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074765.515987:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074765.515996:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 18 pid: 6512 timeout: 1853 lvb_type: 0 00010000:00010000:0.0:1769074765.516007:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 18 pid: 6512 timeout: 1853 lvb_type: 0 00010000:00010000:0.0:1769074765.516023:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd532e4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6c0 expref: 18 pid: 6512 timeout: 1853 lvb_type: 0 00010000:00010000:0.0:1769074765.516036:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074765.516040:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074765.516049:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074765.516056:0:6499:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074765.516089:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074765.516118:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2406:x1855007616766592:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 565us (1067us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.516126:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6911 00010000:00010000:1.0:1769074765.519989:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074765.520007:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074765.520030:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074765.520177:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3c00/0x7fe2323adfd53362 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 236 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.520196:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 235 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.520205:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.520208:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074765.520211:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b64eff3e00, rc 0) 00000100:00100000:1.0:1769074765.520289:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62707:x1855007616766208:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 16757us (17194us total) trans 8589943905 rc 0/0 00000100:00100000:1.0:1769074765.520301:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13298 00000100:00100000:2.0:1769074765.544401:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625104640:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074765.544437:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074765.544496:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616766336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 31122us (33979us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.544507:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6909 00000100:00100000:0.0:1769074765.545948:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.546060:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616766848 00000100:00100000:1.0:1769074765.546073:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616766848 00000100:00100000:1.0:1769074765.546148:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13299 00000100:00100000:1.0:1769074765.546158:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+25:62709:x1855007616766848:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074765.546188:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.546215:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53377 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b722 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.546370:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074765.546387:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074765.546399:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074765.546410:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074765.546547:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53377 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 28 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.546565:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 27 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.546573:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074765.546576:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074765.546580:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8400, rc 0) 00000100:00100000:1.0:1769074765.546645:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+27:62709:x1855007616766848:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 489us (708us total) trans 8589943906 rc 0/0 00000100:00100000:1.0:1769074765.546654:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13299 00000100:00100000:0.0:1769074765.584174:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.588223:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616766976 00000100:00100000:2.0:1769074765.588240:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616766976 00002000:00100000:2.0:1769074765.588318:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547599500 x1855007616766976/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.588330:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599500 with x1855007616766976 ext(1048576->2097151) 00002000:00010000:2.0:1769074765.588336:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547599500 00000100:00100000:2.0:1769074765.588377:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8406 00000100:00100000:2.0:1769074765.588387:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2407:x1855007616766976:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074765.599172:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.599261:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767104 00000100:00100000:3.0:1769074765.599273:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767104 00002000:00100000:3.0:1769074765.599317:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546f94700 x1855007616767104/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.599329:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94700 with x1855007616767104 ext(1048576->2097151) 00002000:00010000:3.0:1769074765.599336:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f94700 00000100:00100000:3.0:1769074765.599366:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8407 00000100:00100000:3.0:1769074765.599374:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2407:x1855007616767104:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074765.614296:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2407:x1855007616766976:12345-192.168.204.60@tcp:3:cat.0 Request processed in 25897us (30120us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074765.614312:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8406 00002000:00100000:2.0:1769074765.614326:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547599500 x1855007616766976/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.614337:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547599500 with x1855007616766976 ext(1048576->2097151) 00002000:00010000:2.0:1769074765.614342:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547599500 00000100:00100000:2.0:1769074765.616440:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074765.616907:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767232 00000100:00100000:0.0:1769074765.616918:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767232 00000100:00100000:0.0:1769074765.616973:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13899 00000100:00100000:0.0:1769074765.616985:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616767232:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074765.617002:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074765.617019:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b729 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.617043:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074765.617046:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074765.617055:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 22 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074765.617072:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 22 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074765.617084:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:0.0:1769074765.617148:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616767232:12345-192.168.204.60@tcp:101:cat.0 Request processed in 165us (728us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.617155:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13899 00000100:00100000:3.0:1769074765.643941:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2407:x1855007616767104:12345-192.168.204.60@tcp:3:cat.0 Request processed in 44550us (44762us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074765.643958:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8407 00002000:00100000:3.0:1769074765.643973:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546f94700 x1855007616767104/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.643984:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94700 with x1855007616767104 ext(1048576->2097151) 00002000:00010000:3.0:1769074765.643992:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f94700 00000100:00100000:1.0:1769074765.664081:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.664236:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767360 00000100:00100000:3.0:1769074765.664258:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767360 00002000:00100000:3.0:1769074765.664309:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547ef4a80 x1855007616767360/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.664328:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef4a80 with x1855007616767360 ext(1048576->2097151) 00002000:00010000:3.0:1769074765.664334:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef4a80 00000100:00100000:3.0:1769074765.664369:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8408 00000100:00100000:3.0:1769074765.664378:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616767360:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074765.684377:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616767360:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19987us (20294us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074765.684392:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8408 00002000:00100000:3.0:1769074765.684405:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547ef4a80 x1855007616767360/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/440 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.684418:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef4a80 with x1855007616767360 ext(1048576->2097151) 00002000:00010000:3.0:1769074765.684434:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef4a80 00000100:00100000:0.0:1769074765.841468:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.841628:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767488 00000100:00100000:2.0:1769074765.841642:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767488 00000100:00100000:2.0:1769074765.841708:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13900 00000100:00100000:2.0:1769074765.841718:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62709:x1855007616767488:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074765.841736:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.841753:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b730 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.841784:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.841799:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 23 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.841830:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.841832:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.841836:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [2048000->2097151] (req 2048000->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.841869:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 23 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.841882:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 23 pid: 10284 timeout: 1853 lvb_type: 1 00000100:00100000:2.0:1769074765.841896:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625104768:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.842030:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.842040:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.842049:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625104896:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074765.842071:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.866957:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.866998:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.867413:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767616 00000100:00100000:3.0:1769074765.867453:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767616 00010000:00010000:3.0:1769074765.867497:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 34 pid: 10284 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074765.867540:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767744 00000100:00100000:3.0:1769074765.867544:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767744 00010000:00010000:3.0:1769074765.867557:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 24 pid: 10284 timeout: 1853 lvb_type: 1 00000100:00100000:3.0:1769074765.867588:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6912 00000100:00100000:3.0:1769074765.867600:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2408:x1855007616767616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074765.867614:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074765.867638:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 34 pid: 10284 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.867649:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 34 pid: 10284 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074765.867678:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074765.867734:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2408:x1855007616767616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 136us (786us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074765.867741:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6912 00000100:00100000:3.0:1769074765.867758:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6913 00000100:00100000:3.0:1769074765.867765:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2407:x1855007616767744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074765.867785:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074765.867793:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 24 pid: 10284 timeout: 1853 lvb_type: 1 00010000:00010000:3.0:1769074765.867840:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 24 pid: 10284 timeout: 1853 lvb_type: 1 00010000:00010000:3.0:1769074765.867858:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.867861:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074765.867871:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.867881:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074765.867918:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074765.867931:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625105024:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074765.867981:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00010000:00010000:2.0:1769074765.868765:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd5338c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b729 expref: 24 pid: 10284 timeout: 1853 lvb_type: 1 00000100:00100000:2.0:1769074765.868798:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625104768:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.868850:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5335b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b714 expref: 33 pid: 10284 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.868860:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625104896:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.868889:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.868899:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:2.0:1769074765.868955:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62709:x1855007616767488:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 27237us (27496us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.868963:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13900 00000100:00100000:3.0:1769074765.872859:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625105024:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074765.872886:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074765.872936:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616767744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5170us (5935us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074765.872946:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6913 00000100:00100000:0.0:1769074765.892865:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074765.893024:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616767872 00000100:00100000:3.0:1769074765.893039:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616767872 00002000:00100000:3.0:1769074765.893092:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (2048000->2052095) req@ffff89b54759aa00 x1855007616767872/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.893103:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759aa00 with x1855007616767872 ext(2048000->2052095) 00002000:00010000:3.0:1769074765.893109:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759aa00 00000100:00100000:3.0:1769074765.893144:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8409 00000100:00100000:3.0:1769074765.893154:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616767872:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074765.893316:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616767872:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 164us (462us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074765.893324:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8409 00002000:00100000:3.0:1769074765.893331:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (2048000->2052095) req@ffff89b54759aa00 x1855007616767872/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/4536 e 0 to 0 dl 1769074776 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074765.893345:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759aa00 with x1855007616767872 ext(2048000->2052095) 00002000:00010000:3.0:1769074765.893349:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759aa00 00000100:00100000:0.0:1769074765.900451:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.900801:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616768128 00000100:00100000:2.0:1769074765.900838:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616768128 00000100:00100000:2.0:1769074765.900907:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13300 00000100:00100000:2.0:1769074765.900917:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62718:x1855007616768128:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074765.900940:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.900960:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5339a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b737 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.901042:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901057:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901072:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901087:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901093:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901100:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901211:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901222:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901230:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074765.901255:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074765.901352:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901360:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901366:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901372:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901379:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641650800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074765.901384:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901391:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901404:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074765.901408:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074765.901412:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901418:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd533a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901567:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901575:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901671:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901676:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641650a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074765.901680:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901686:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901694:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4400) 00010000:00010000:2.0:1769074765.901697:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074765.901700:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901709:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd533a8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074765.901760:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd5339a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 238 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.901776:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 237 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.901784:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074765.901786:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074765.901789:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fa3c800, rc 0) 00000100:00100000:2.0:1769074765.902000:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62718:x1855007616768128:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1083us (1577us total) trans 8589943907 rc 0/0 00000100:00100000:2.0:1769074765.902010:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13300 00000100:00100000:1.0:1769074765.924693:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.924794:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616768256 00000100:00100000:2.0:1769074765.924838:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616768256 00000100:00100000:2.0:1769074765.924887:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13901 00000100:00100000:2.0:1769074765.924894:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62709:x1855007616768256:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074765.924912:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074765.924933:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b73e expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.924966:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.924976:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 32 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.924984:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.924986:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074765.924990:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1781759] (req 1048576->1781759) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.925024:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 32 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074765.925036:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 32 pid: 9048 timeout: 1853 lvb_type: 1 00000100:00100000:2.0:1769074765.925050:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625105152:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.925145:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.925153:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074765.925162:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625105280:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074765.925184:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074765.931676:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.931882:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616768512 00000100:00100000:2.0:1769074765.931896:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616768512 00000100:00100000:2.0:1769074765.931968:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4648 00000100:00100000:2.0:1769074765.931980:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62718:x1855007616768512:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:2.0:1769074765.932402:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+238:62718:x1855007616768512:12345-192.168.204.60@tcp:35:cat.0 Request processed in 420us (737us total) trans 8589943908 rc 0/0 00000100:00100000:2.0:1769074765.932411:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4648 00000100:00100000:2.0:1769074765.942568:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625105152:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074765.942592:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625105280:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074765.942630:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1781759] (req 1048576->1781759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074765.942649:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651e00, rc 0) 00000100:00100000:2.0:1769074765.942710:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62709:x1855007616768256:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17815us (18025us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.942720:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13901 00000100:00100000:1.0:1769074765.949076:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.949125:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.949151:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.949178:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.949205:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074765.949235:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616768768 00000100:00100000:1.0:1769074765.949261:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616768768 00010000:00010000:1.0:1769074765.949310:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 33 pid: 9048 timeout: 1853 lvb_type: 1 00000100:00100000:1.0:1769074765.949343:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769152 00000100:00100000:1.0:1769074765.949346:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769152 00010000:00010000:1.0:1769074765.949359:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:1.0:1769074765.949378:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6914 00000100:00100000:1.0:1769074765.949388:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2407:x1855007616768768:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.949402:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.949435:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 33 pid: 9048 timeout: 1853 lvb_type: 1 00010000:00010000:1.0:1769074765.949460:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 33 pid: 9048 timeout: 1853 lvb_type: 1 00010000:00010000:1.0:1769074765.949485:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd53354 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b70d expref: 33 pid: 9048 timeout: 1853 lvb_type: 1 00010000:00010000:1.0:1769074765.949506:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074765.949570:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616768768:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 183us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074765.949589:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6914 00000100:00100000:1.0:1769074765.949646:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6915 00000100:00100000:1.0:1769074765.949652:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616769152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074765.949660:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074765.949668:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074765.949677:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074765.949690:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd53300 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6e3 expref: 23 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074765.949699:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.949701:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074765.949709:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.949719:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074765.949776:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074765.949790:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625105408:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074765.949901:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074765.950706:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616768640 00000100:00100000:0.0:1769074765.950715:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616768640 00000100:00100000:0.0:1769074765.950757:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616768896 00000100:00100000:0.0:1769074765.950760:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616768896 00000100:00100000:0.0:1769074765.950789:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769024 00000100:00100000:0.0:1769074765.950792:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769024 00000100:00100000:0.0:1769074765.950857:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13902 00000100:00100000:0.0:1769074765.950867:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:62707:x1855007616768640:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074765.950883:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074765.950898:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b745 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.950920:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.950929:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074765.950931:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074765.950934:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1556480->2097151] (req 1556480->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 35 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.950961:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.950976:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:0.0:1769074765.950989:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625105536:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074765.951045:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074765.951069:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13903 00000100:00100000:0.0:1769074765.951077:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616768896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074765.951085:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074765.951093:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533cb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b74c expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074765.951124:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625105664:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074765.951150:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074765.952026:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13904 00000100:00100000:1.0:1769074765.952042:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616769024:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074765.952056:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074765.952070:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd533d2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b753 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074765.952107:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625105792:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074765.952175:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074765.961481:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625105408:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074765.961511:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074765.961593:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2408:x1855007616769152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11927us (12373us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074765.961606:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6915 00000100:00100000:0.0:1769074765.961991:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625105536:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074765.962018:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1556480->2097151] (req 1556480->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 37 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074765.962043:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:0.0:1769074765.962090:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:62707:x1855007616768640:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11225us (13026us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074765.962098:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13902 00002000:00010000:3.0:1769074765.962359:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074764 -> 1769074765 00002000:00010000:3.0:1769074765.962363:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074764 -> 1769074765 00000100:00100000:3.0:1769074765.962382:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625105664:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074765.962406:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533cb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b74c expref: 36 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.962433:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533cb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b74c expref: 36 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.962448:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:3.0:1769074765.962485:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616768896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11409us (13333us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074765.962492:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13903 00002000:00010000:3.0:1769074765.972215:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074764 -> 1769074765 00002000:00010000:3.0:1769074765.972228:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074764 -> 1769074765 00002000:00010000:3.0:1769074765.972230:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074764 -> 1769074765 00000100:00100000:3.0:1769074765.972273:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625105792:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074765.972318:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd533d2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b753 expref: 34 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.972348:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd533d2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b753 expref: 34 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074765.972368:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8a00, rc 0) 00000100:00100000:3.0:1769074765.972470:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616769024:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20433us (23289us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074765.972480:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13904 00000100:00100000:1.0:1769074765.988217:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074765.988407:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769280 00000100:00100000:2.0:1769074765.988433:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769280 00002000:00100000:2.0:1769074765.988515:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1777664->1781759) req@ffff89b54441df80 x1855007616769280/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/0 e 0 to 0 dl 1769074776 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.988529:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441df80 with x1855007616769280 ext(1777664->1781759) 00010000:00010000:2.0:1769074765.988539:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.988559:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074765.988568:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441df80 00000100:00100000:2.0:1769074765.988625:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 719 00000100:00100000:2.0:1769074765.988638:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2408:x1855007616769280:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074765.989196:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2408:x1855007616769280:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 558us (989us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074765.989205:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 719 00002000:00100000:2.0:1769074765.989212:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1777664->1781759) req@ffff89b54441df80 x1855007616769280/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:301/0 lens 488/4536 e 0 to 0 dl 1769074776 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074765.989221:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441df80 with x1855007616769280 ext(1777664->1781759) 00010000:00010000:2.0:1769074765.989227:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074765.989252:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074765.989260:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54441df80 00000100:00100000:1.0:1769074766.014349:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.014397:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.014499:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.014997:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769536 00000100:00100000:2.0:1769074766.015011:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769536 00002000:00100000:2.0:1769074766.015074:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1781759) req@ffff89b54b427100 x1855007616769536/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.015086:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427100 with x1855007616769536 ext(1048576->1781759) 00010000:00010000:2.0:1769074766.015096:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.015118:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 22 pid: 8398 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074766.015125:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54b427100 00000100:00100000:2.0:1769074766.015167:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 720 00000100:00100000:2.0:1769074766.015177:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616769536:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074766.015406:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769408 00000100:00100000:0.0:1769074766.015411:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769408 00000100:00100000:0.0:1769074766.015892:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769664 00000100:00100000:0.0:1769074766.015898:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769664 00000100:00100000:0.0:1769074766.015922:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13905 00000100:00100000:0.0:1769074766.015930:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:62718:x1855007616769408:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.015941:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.015961:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b75a expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.015989:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.015997:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074766.016000:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074766.016004:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 34 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.016046:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.016057:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 24 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:0.0:1769074766.016071:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625105920:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074766.016137:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074766.016169:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13906 00000100:00100000:0.0:1769074766.016177:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2406:x1855007616769664:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.016184:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.016192:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd533e0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b761 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074766.016223:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625106048:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074766.016248:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00080000:2.0:1769074766.023444:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074766.032036:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625105920:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074766.032076:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 34 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.032098:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:0.0:1769074766.032169:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:62718:x1855007616769408:12345-192.168.204.60@tcp:101:cat.0 Request processed in 16239us (17830us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074766.032182:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13905 00000100:00100000:2.0:1769074766.035397:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2409:x1855007616769536:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 20209us (20986us total) trans 21474841178 rc 0/0 00000100:00100000:2.0:1769074766.035414:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 720 00002000:00100000:2.0:1769074766.035439:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1781759) req@ffff89b54b427100 x1855007616769536/t21474841178(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.035450:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427100 with x1855007616769536 ext(1048576->1781759) 00010000:00010000:2.0:1769074766.035460:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 26 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.035474:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 26 pid: 8398 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074766.035482:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54b427100 00000100:00100000:3.0:1769074766.038060:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.038153:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769792 00000100:00100000:1.0:1769074766.038164:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769792 00002000:00100000:1.0:1769074766.038217:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (2048000->2097151) req@ffff89b54d101880 x1855007616769792/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.038229:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101880 with x1855007616769792 ext(2048000->2097151) 00010000:00010000:1.0:1769074766.038239:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 26 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074766.038258:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 26 pid: 8398 timeout: 1853 lvb_type: 0 00002000:00010000:1.0:1769074766.038266:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d101880 00000100:00100000:1.0:1769074766.038302:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 721 00000100:00100000:1.0:1769074766.038313:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2408:x1855007616769792:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074766.038973:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625106048:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074766.039007:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd533e0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b761 expref: 27 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074766.039035:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052e00/0x7fe2323adfd533e0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b761 expref: 27 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074766.039048:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052e00, rc 0) 00000100:00100000:2.0:1769074766.039099:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2406:x1855007616769664:12345-192.168.204.60@tcp:101:cat.0 Request processed in 22922us (24600us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074766.039109:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13906 00000001:00080000:0.0:1769074766.045643:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841178 is committed 00000001:00080000:0.0:1769074766.045665:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841176 is committed 00000001:00080000:0.0:1769074766.045693:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841168 is committed 00000100:00100000:2.0:1769074766.047286:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616769792:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 8964us (9225us total) trans 21474841179 rc 0/0 00000100:00100000:2.0:1769074766.047300:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 721 00002000:00100000:2.0:1769074766.047314:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (2048000->2097151) req@ffff89b54d101880 x1855007616769792/t21474841179(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.047325:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101880 with x1855007616769792 ext(2048000->2097151) 00010000:00010000:2.0:1769074766.047336:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.047348:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00002000:00010000:2.0:1769074766.047357:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d101880 00000100:00100000:1.0:1769074766.048173:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.048278:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.048958:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616769920 00000100:00100000:2.0:1769074766.048968:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616769920 00010000:00010000:2.0:1769074766.049007:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074766.049048:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6916 00000100:00100000:2.0:1769074766.049057:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616769920:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074766.049072:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074766.049092:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.049103:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.049126:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd533bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1781759) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b73e expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.049142:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074766.049145:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074766.049152:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.049162:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.049203:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074766.049217:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625106176:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074766.049306:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.049536:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770176 00000100:00100000:3.0:1769074766.049543:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770176 00000100:00100000:3.0:1769074766.049592:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13907 00000100:00100000:3.0:1769074766.049599:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62721:x1855007616770176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074766.049614:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.049630:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b768 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.049645:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074766.049650:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.049660:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.049670:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5800, rc 0) 00000100:00100000:3.0:1769074766.049698:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62721:x1855007616770176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 100us (1420us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.049705:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13907 00000100:00100000:2.0:1769074766.054524:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f94e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625106176:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074766.054556:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074766.054627:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616769920:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5570us (6458us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074766.054639:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6916 00000100:00100000:0.0:1769074766.057025:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.057071:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.057150:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770304 00000100:00100000:2.0:1769074766.057161:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770304 00002000:00100000:2.0:1769074766.057215:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1556480->1560575) req@ffff89b5444a5c00 x1855007616770304/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.057226:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5c00 with x1855007616770304 ext(1556480->1560575) 00002000:00010000:2.0:1769074766.057232:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5c00 00000100:00100000:2.0:1769074766.057269:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8410 00000100:00100000:2.0:1769074766.057280:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616770304:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074766.057489:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616770304:12345-192.168.204.60@tcp:3:cat.0 Request processed in 210us (475us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074766.057496:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8410 00002000:00100000:2.0:1769074766.057502:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1556480->1560575) req@ffff89b5444a5c00 x1855007616770304/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.057510:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5c00 with x1855007616770304 ext(1556480->1560575) 00002000:00010000:2.0:1769074766.057513:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5c00 00000100:00100000:1.0:1769074766.057602:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770432 00000100:00100000:1.0:1769074766.057606:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770432 00010000:00010000:1.0:1769074766.057631:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:1.0:1769074766.057669:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6917 00000100:00100000:1.0:1769074766.057677:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616770432:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.057689:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074766.057707:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074766.057720:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074766.057743:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53393 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2048000->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b730 expref: 12 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:1.0:1769074766.057764:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.057766:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.057773:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.057782:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.057790:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.057792:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.057795:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.057802:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.057874:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.057891:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625106304:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.057976:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.057986:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625106432:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.058010:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074766.063156:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625106304:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.063173:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625106432:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.063199:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.063282:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616770432:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5603us (6208us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.063294:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6917 00000100:00100000:0.0:1769074766.076118:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.076219:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770560 00000100:00100000:3.0:1769074766.076232:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770560 00002000:00100000:3.0:1769074766.076294:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b549f28380 x1855007616770560/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.076306:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28380 with x1855007616770560 ext(1048576->2097151) 00002000:00010000:3.0:1769074766.076313:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f28380 00000100:00100000:3.0:1769074766.076356:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8411 00000100:00100000:3.0:1769074766.076366:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616770560:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074766.092723:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.092868:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770688 00000100:00100000:3.0:1769074766.092879:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770688 00000100:00100000:3.0:1769074766.092940:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13908 00000100:00100000:3.0:1769074766.092952:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62707:x1855007616770688:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074766.092975:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.093001:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b76f expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.093036:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.093047:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074766.093055:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.093063:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074766.093065:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074766.093072:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->1380351] (req 1048576->1380351) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.093106:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.093120:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 18 pid: 9048 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074766.093133:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625106560:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074766.093245:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074766.093255:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 11 pid: 28955 timeout: 1853 lvb_type: 1 00000100:00100000:3.0:1769074766.093264:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625106688:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074766.093298:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.093307:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 24 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:3.0:1769074766.093316:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625106816:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074766.093340:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bad780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074766.103034:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.103147:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.104120:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616770560:12345-192.168.204.60@tcp:3:cat.0 Request processed in 27747us (28008us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074766.104146:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8411 00002000:00100000:3.0:1769074766.104164:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b549f28380 x1855007616770560/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.104174:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28380 with x1855007616770560 ext(1048576->2097151) 00002000:00010000:3.0:1769074766.104182:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f28380 00000100:00100000:1.0:1769074766.104261:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770816 00000100:00100000:1.0:1769074766.104274:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770816 00002000:00100000:1.0:1769074766.104324:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546f97480 x1855007616770816/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.104332:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97480 with x1855007616770816 ext(1048576->2097151) 00002000:00010000:1.0:1769074766.104336:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f97480 00000100:00100000:1.0:1769074766.104372:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616770944 00000100:00100000:1.0:1769074766.104377:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616770944 00002000:00100000:1.0:1769074766.104393:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1556479) req@ffff89b546f95c00 x1855007616770944/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.104401:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95c00 with x1855007616770944 ext(1048576->1556479) 00002000:00010000:1.0:1769074766.104404:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f95c00 00000100:00100000:1.0:1769074766.104435:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8412 00000100:00100000:1.0:1769074766.104444:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616770816:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074766.109918:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8413 00000100:00100000:1.0:1769074766.109934:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2406:x1855007616770944:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074766.128287:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.128314:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.128372:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771072 00000100:00100000:2.0:1769074766.128384:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771072 00010000:00010000:2.0:1769074766.128453:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 19 pid: 9048 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074766.128491:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771200 00000100:00100000:2.0:1769074766.128494:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771200 00010000:00010000:2.0:1769074766.128516:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 12 pid: 28955 timeout: 1853 lvb_type: 1 00000100:00100000:2.0:1769074766.128536:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6918 00000100:00100000:2.0:1769074766.128546:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616771072:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074766.128566:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074766.128586:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 19 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.128601:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 19 pid: 9048 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.128631:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074766.128689:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616771072:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 144us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074766.128697:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6918 00000100:00100000:2.0:1769074766.128714:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6919 00000100:00100000:2.0:1769074766.128721:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616771200:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074766.128728:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074766.128735:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 12 pid: 28955 timeout: 1853 lvb_type: 1 00010000:00010000:2.0:1769074766.128745:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 12 pid: 28955 timeout: 1853 lvb_type: 1 00010000:00010000:2.0:1769074766.128756:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074766.128783:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616771200:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 62us (468us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074766.128790:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6919 00000100:00080000:2.0:1769074766.128832:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074766.128864:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.128882:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771328 00000100:00100000:2.0:1769074766.128886:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771328 00010000:00010000:2.0:1769074766.128900:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 25 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:2.0:1769074766.128918:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6920 00000100:00100000:2.0:1769074766.128924:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616771328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074766.128934:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074766.128941:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 25 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.128951:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 25 pid: 8398 timeout: 1853 lvb_type: 0 00010000:00010000:2.0:1769074766.128966:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074766.128968:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074766.128975:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.128985:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.129025:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074766.129042:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625106944:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074766.129097:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260480 going to sleep for 11 seconds 00010000:00010000:0.0:1769074766.130630:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cf6c600/0x7fe2323adfd530f3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b5af expref: 18 pid: 9048 timeout: 1853 lvb_type: 0 00000100:00100000:0.0:1769074766.130653:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625106560:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074766.130669:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd5320b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b650 expref: 11 pid: 28955 timeout: 1853 lvb_type: 1 00000100:00100000:0.0:1769074766.130678:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625106688:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074766.130690:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd53091 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b577 expref: 25 pid: 8398 timeout: 1853 lvb_type: 0 00000100:00100000:0.0:1769074766.130699:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625106816:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074766.130731:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.130744:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5600, rc 0) 00000100:00100000:0.0:1769074766.131268:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62707:x1855007616770688:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 38311us (38552us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074766.131282:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13908 00000100:00100000:0.0:1769074766.135334:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625106944:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074766.135377:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074766.135451:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616771328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6526us (6585us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074766.135463:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6920 00000100:00100000:3.0:1769074766.136288:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2406:x1855007616770944:12345-192.168.204.60@tcp:3:cat.0 Request processed in 26353us (33135us total) trans 0 rc 507904/507904 00000100:00100000:3.0:1769074766.136301:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8413 00002000:00100000:3.0:1769074766.136314:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1556479) req@ffff89b546f95c00 x1855007616770944/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 507904/507904 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.136325:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95c00 with x1855007616770944 ext(1048576->1556479) 00002000:00010000:3.0:1769074766.136333:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f95c00 00000100:00100000:1.0:1769074766.137194:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2406:x1855007616770816:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32745us (34165us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074766.137211:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8412 00002000:00100000:1.0:1769074766.137222:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b546f97480 x1855007616770816/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.137510:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97480 with x1855007616770816 ext(1048576->2097151) 00002000:00010000:1.0:1769074766.137520:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f97480 00000100:00100000:2.0:1769074766.149049:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.149166:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771456 00000100:00100000:0.0:1769074766.149177:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771456 00000100:00100000:0.0:1769074766.149247:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13909 00000100:00100000:0.0:1769074766.149257:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62721:x1855007616771456:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.149275:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.149294:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b776 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.149326:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.149335:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074766.149337:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074766.149341:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.149374:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 15 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.149389:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 15 pid: 10284 timeout: 1853 lvb_type: 0 00000100:00100000:0.0:1769074766.149405:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625107072:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074766.149498:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.152151:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625107072:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074766.152194:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.152217:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:3.0:1769074766.152299:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62721:x1855007616771456:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3043us (3261us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.152311:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13909 00000100:00100000:2.0:1769074766.161756:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.161977:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771584 00000100:00100000:1.0:1769074766.161992:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771584 00002000:00100000:1.0:1769074766.162051:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1376256->1380351) req@ffff89b54d0e5880 x1855007616771584/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.162062:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5880 with x1855007616771584 ext(1376256->1380351) 00002000:00010000:1.0:1769074766.162068:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e5880 00000100:00100000:1.0:1769074766.162098:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8414 00000100:00100000:1.0:1769074766.162108:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616771584:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074766.162325:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616771584:12345-192.168.204.60@tcp:3:cat.0 Request processed in 218us (580us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074766.162333:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8414 00002000:00100000:1.0:1769074766.162340:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1376256->1380351) req@ffff89b54d0e5880 x1855007616771584/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.162348:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e5880 with x1855007616771584 ext(1376256->1380351) 00002000:00010000:1.0:1769074766.162350:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e5880 00000100:00100000:2.0:1769074766.165762:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.165870:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771840 00000100:00100000:3.0:1769074766.165881:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771840 00000100:00100000:3.0:1769074766.165938:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13910 00000100:00100000:3.0:1769074766.165948:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:62718:x1855007616771840:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074766.165968:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.165988:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b77d expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.166016:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074766.166022:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.166033:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.166045:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b4000, rc 0) 00000100:00100000:3.0:1769074766.166103:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62718:x1855007616771840:12345-192.168.204.60@tcp:101:cat.0 Request processed in 156us (350us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.166110:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13910 00000100:00100000:0.0:1769074766.172594:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.172681:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771712 00000100:00100000:3.0:1769074766.172691:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771712 00002000:00100000:3.0:1769074766.172736:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1556480->2097151) req@ffff89b549f28e00 x1855007616771712/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.172747:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28e00 with x1855007616771712 ext(1556480->2097151) 00010000:00010000:3.0:1769074766.172756:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 17 pid: 10284 timeout: 1853 lvb_type: 0 00010000:00010000:3.0:1769074766.172780:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 17 pid: 10284 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074766.172788:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549f28e00 00000100:00100000:3.0:1769074766.172866:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 722 00000100:00100000:3.0:1769074766.172875:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616771712:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074766.184137:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616771712:12345-192.168.204.60@tcp:4:cat.0 Request processed in 11247us (11538us total) trans 21474841180 rc 0/0 00000100:00100000:3.0:1769074766.184154:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 722 00002000:00100000:3.0:1769074766.184171:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1556480->2097151) req@ffff89b549f28e00 x1855007616771712/t21474841180(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.184183:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f28e00 with x1855007616771712 ext(1556480->2097151) 00010000:00010000:3.0:1769074766.184196:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 19 pid: 10284 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.184220:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 19 pid: 10284 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074766.184230:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549f28e00 00000100:00100000:2.0:1769074766.194982:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.195470:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616771968 00000100:00100000:0.0:1769074766.195480:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616771968 00010000:00010000:0.0:1769074766.195525:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 19 pid: 10284 timeout: 1854 lvb_type: 0 00000100:00100000:0.0:1769074766.195573:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6921 00000100:00100000:0.0:1769074766.195584:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616771968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074766.195600:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074766.195621:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 19 pid: 10284 timeout: 1854 lvb_type: 0 00010000:00010000:0.0:1769074766.195634:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 19 pid: 10284 timeout: 1854 lvb_type: 0 00010000:00010000:0.0:1769074766.195659:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd533c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1556480->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b745 expref: 19 pid: 10284 timeout: 1854 lvb_type: 0 00010000:00010000:0.0:1769074766.195677:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.195679:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.195688:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.195704:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.195712:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.195714:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.195718:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.195725:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.195763:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074766.195778:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625107200:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.196214:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.196231:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625107328:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.196259:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074766.201057:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625107200:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.207919:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625107328:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074766.207941:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074766.207966:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.208035:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616771968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12451us (13063us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.208047:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6921 00000100:00100000:3.0:1769074766.208119:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772096 00000100:00100000:3.0:1769074766.208140:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772096 00002000:00100000:3.0:1769074766.208196:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e6300 x1855007616772096/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.208207:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6300 with x1855007616772096 ext(1048576->2097151) 00002000:00010000:3.0:1769074766.208212:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e6300 00000100:00100000:3.0:1769074766.208243:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8415 00000100:00100000:3.0:1769074766.208251:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616772096:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074766.234195:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.234256:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.234513:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772352 00000100:00100000:1.0:1769074766.234525:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772352 00002000:00100000:1.0:1769074766.234590:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1380352->2097151) req@ffff89b54d0e4a80 x1855007616772352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.234603:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4a80 with x1855007616772352 ext(1380352->2097151) 00002000:00010000:1.0:1769074766.234610:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e4a80 00000100:00100000:1.0:1769074766.234639:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772224 00000100:00100000:1.0:1769074766.234642:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772224 00002000:00100000:1.0:1769074766.234659:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e4380 x1855007616772224/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.234669:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4380 with x1855007616772224 ext(1048576->2097151) 00002000:00010000:1.0:1769074766.234672:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e4380 00000100:00100000:1.0:1769074766.234693:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8416 00000100:00100000:1.0:1769074766.234702:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616772352:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074766.235758:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8417 00000100:00100000:1.0:1769074766.235784:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616772224:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074766.235911:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616772096:12345-192.168.204.60@tcp:3:cat.0 Request processed in 27658us (27969us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074766.235920:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8415 00002000:00100000:0.0:1769074766.235929:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e6300 x1855007616772096/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074766.235937:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e6300 with x1855007616772096 ext(1048576->2097151) 00002000:00010000:0.0:1769074766.235942:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e6300 00010000:00010000:0.0:1769074766.239208:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074766.248170:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616772352:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13450us (13966us total) trans 0 rc 716800/716800 00000100:00100000:1.0:1769074766.248192:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8416 00002000:00100000:1.0:1769074766.248212:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1380352->2097151) req@ffff89b54d0e4a80 x1855007616772352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 716800/716800 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.248224:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4a80 with x1855007616772352 ext(1380352->2097151) 00002000:00010000:1.0:1769074766.248232:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d0e4a80 00000100:00100000:0.0:1769074766.257956:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.258402:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772480 00000100:00100000:3.0:1769074766.258423:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772480 00000100:00100000:3.0:1769074766.258499:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13911 00000100:00100000:3.0:1769074766.258510:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62721:x1855007616772480:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074766.258530:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.258552:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b784 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.258581:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.258589:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074766.258592:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074766.258596:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.258627:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.258640:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 22 pid: 28955 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074766.258653:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625107456:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074766.258723:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544bade40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074766.259880:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616772224:12345-192.168.204.60@tcp:3:cat.0 Request processed in 24092us (25615us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074766.259904:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8417 00002000:00100000:1.0:1769074766.259917:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54d0e4380 x1855007616772224/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.259928:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e4380 with x1855007616772224 ext(1048576->2097151) 00002000:00010000:1.0:1769074766.259935:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e4380 00000100:00100000:0.0:1769074766.268520:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.270379:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.270505:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772736 00000100:00100000:0.0:1769074766.270515:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772736 00000100:00100000:0.0:1769074766.270573:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13912 00000100:00100000:0.0:1769074766.270583:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62718:x1855007616772736:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.270601:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.270620:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b78b expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.270650:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074766.270657:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.270669:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.270681:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867c00, rc 0) 00000100:00100000:0.0:1769074766.270736:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62718:x1855007616772736:12345-192.168.204.60@tcp:101:cat.0 Request processed in 156us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074766.270756:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13912 00000100:00100000:1.0:1769074766.271553:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625107456:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.271588:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.271602:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f71400, rc 0) 00000100:00100000:1.0:1769074766.271654:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62721:x1855007616772480:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13147us (13713us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.271662:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13911 00000100:00100000:2.0:1769074766.272032:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772608 00000100:00100000:2.0:1769074766.272040:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772608 00002000:00100000:2.0:1769074766.272094:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1380351) req@ffff89b549f2b480 x1855007616772608/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.272105:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2b480 with x1855007616772608 ext(1048576->1380351) 00010000:00010000:2.0:1769074766.272113:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 24 pid: 28955 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074766.272137:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 24 pid: 28955 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074766.272152:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f2b480 00000100:00100000:2.0:1769074766.272186:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 723 00000100:00100000:2.0:1769074766.272197:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616772608:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074766.283767:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616772608:12345-192.168.204.60@tcp:4:cat.0 Request processed in 11559us (15245us total) trans 17179874493 rc 0/0 00000100:00100000:2.0:1769074766.283792:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 723 00002000:00100000:2.0:1769074766.283833:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1380351) req@ffff89b549f2b480 x1855007616772608/t17179874493(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.283845:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2b480 with x1855007616772608 ext(1048576->1380351) 00010000:00010000:2.0:1769074766.283855:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 26 pid: 28955 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074766.283873:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 26 pid: 28955 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074766.283881:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f2b480 00000100:00100000:3.0:1769074766.287932:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.288034:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772864 00000100:00100000:1.0:1769074766.288045:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772864 00010000:00010000:1.0:1769074766.288085:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 26 pid: 28955 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.288127:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6922 00000100:00100000:1.0:1769074766.288137:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616772864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.288151:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074766.288168:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074763 -> 1769074765 00002000:00010000:1.0:1769074766.288171:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074763 -> 1769074765 00010000:00010000:1.0:1769074766.288177:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 26 pid: 28955 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.288190:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 26 pid: 28955 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.288212:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5600/0x7fe2323adfd533ee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1380351) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b76f expref: 26 pid: 28955 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.288228:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.288239:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.288247:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.288257:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.288266:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.288269:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.288272:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.288280:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.288314:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.288328:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625107584:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.288407:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.288425:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625107712:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.288449:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074766.291006:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625107584:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.291022:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625107712:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.291045:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.291101:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616772864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 2964us (3177us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.291111:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6922 00000100:00100000:0.0:1769074766.297565:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.297637:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616772992 00000100:00100000:2.0:1769074766.297649:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616772992 00002000:00100000:2.0:1769074766.297696:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b549f2aa00 x1855007616772992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.297707:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2aa00 with x1855007616772992 ext(1048576->2097151) 00002000:00010000:2.0:1769074766.297712:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549f2aa00 00000100:00100000:2.0:1769074766.297748:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8418 00000100:00100000:2.0:1769074766.297757:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616772992:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074766.311354:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.311471:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.312458:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616772992:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14696us (14897us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074766.312472:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8418 00002000:00100000:2.0:1769074766.312484:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b549f2aa00 x1855007616772992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.312493:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2aa00 with x1855007616772992 ext(1048576->2097151) 00002000:00010000:2.0:1769074766.312499:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549f2aa00 00000100:00100000:2.0:1769074766.312526:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773248 00000100:00100000:2.0:1769074766.312533:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773248 00002000:00100000:2.0:1769074766.312569:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54d103b80 x1855007616773248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.312577:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103b80 with x1855007616773248 ext(1048576->2097151) 00002000:00010000:2.0:1769074766.312580:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d103b80 00000100:00100000:2.0:1769074766.312610:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8419 00000100:00100000:2.0:1769074766.312620:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616773248:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074766.315065:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773120 00000100:00100000:1.0:1769074766.315080:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773120 00000100:00100000:1.0:1769074766.315136:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13913 00000100:00100000:1.0:1769074766.315145:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62725:x1855007616773120:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074766.315167:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.315193:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b792 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.315224:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 13 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.315244:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 8 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.315252:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 17 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.315258:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.315260:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.315264:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2609152->2863103] (req 2609152->2863103) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.315295:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 17 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.315308:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 17 pid: 27844 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.315326:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625107840:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.315406:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 8 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.315433:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 8 pid: 27843 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.315441:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625107968:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.315470:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 13 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.315478:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 13 pid: 28955 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.315487:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625108096:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.315505:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074766.338500:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.338537:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.338567:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773376 00000100:00100000:2.0:1769074766.338574:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.338577:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773376 00000100:00100000:2.0:1769074766.338594:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773632 00000100:00100000:2.0:1769074766.338597:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773632 00010000:00010000:3.0:1769074766.338625:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 9 pid: 27843 timeout: 1854 lvb_type: 1 00000100:00100000:2.0:1769074766.338635:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13914 00000100:00100000:2.0:1769074766.338645:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62718:x1855007616773632:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:3.0:1769074766.338648:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773504 00000100:00100000:3.0:1769074766.338650:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773504 00010000:00010000:2.0:1769074766.338659:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.338659:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 19 pid: 27844 timeout: 1854 lvb_type: 1 00000100:00100000:3.0:1769074766.338675:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6923 00010000:00010000:2.0:1769074766.338678:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b799 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074766.338680:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2406:x1855007616773376:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.338694:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074766.338700:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074766.338707:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2609152->2863103] (req 2609152->2863103) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.338714:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074766.338719:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074766.338764:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6924 00000100:00100000:0.0:1769074766.338774:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616773504:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074766.338782:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1769074766.339131:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625107840:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.339157:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625107968:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.339194:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625108096:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074766.339317:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 9 pid: 27843 timeout: 1854 lvb_type: 1 00010000:00010000:3.0:1769074766.339329:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 9 pid: 27843 timeout: 1854 lvb_type: 1 00010000:00010000:3.0:1769074766.339348:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd52e84 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b42e expref: 9 pid: 27843 timeout: 1854 lvb_type: 1 00010000:00010000:3.0:1769074766.339365:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.339439:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2406:x1855007616773376:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 757us (950us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.339447:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6923 00000100:00100000:3.0:1769074766.339507:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.339531:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773760 00000100:00100000:3.0:1769074766.339537:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773760 00010000:00010000:3.0:1769074766.339554:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 14 pid: 28955 timeout: 1854 lvb_type: 1 00000100:00100000:3.0:1769074766.339758:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6925 00000100:00100000:3.0:1769074766.339767:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616773760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.339776:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074766.339786:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 14 pid: 28955 timeout: 1854 lvb_type: 1 00010000:00010000:3.0:1769074766.339894:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 14 pid: 28955 timeout: 1854 lvb_type: 1 00010000:00010000:3.0:1769074766.339911:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd52dc0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3a2 expref: 14 pid: 28955 timeout: 1854 lvb_type: 1 00010000:00010000:3.0:1769074766.339921:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.339942:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616773760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 177us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.339949:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6925 00000100:00080000:3.0:1769074766.340486:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074766.340607:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616773248:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 27987us (29135us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074766.340616:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8419 00002000:00100000:3.0:1769074766.340625:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54d103b80 x1855007616773248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.340634:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103b80 with x1855007616773248 ext(1048576->2097151) 00002000:00010000:3.0:1769074766.340638:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d103b80 00010000:00010000:0.0:1769074766.340734:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 20 pid: 27844 timeout: 1854 lvb_type: 1 00010000:00010000:0.0:1769074766.340753:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 20 pid: 27844 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.341591:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2609152->2863103] (req 2609152->2863103) gid 0 flags: 0x40000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.341608:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:1.0:1769074766.341653:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62725:x1855007616773120:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 26510us (30314us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.341658:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13913 00010000:00010000:0.0:1769074766.356472:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd52dea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3cc expref: 20 pid: 27844 timeout: 1854 lvb_type: 1 00010000:00010000:2.0:1769074766.356481:0:8396:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2609152->2863103] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.356495:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.356497:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.356616:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.356627:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074766.359721:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.359847:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616773888 00000100:00100000:0.0:1769074766.359859:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616773888 00000100:00100000:0.0:1769074766.359915:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13915 00000100:00100000:0.0:1769074766.359925:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62721:x1855007616773888:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.359944:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.359964:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7a0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.360305:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 19 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074766.360326:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:2.0:1769074766.360398:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:62718:x1855007616773632:12345-192.168.204.60@tcp:101:cat.0 Request processed in 21754us (21821us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074766.360406:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13914 00010000:00010000:1.0:1769074766.360750:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.360988:0:6500:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.361006:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625108352:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.362023:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0180 going to sleep for 11 seconds 00010000:00010000:0.0:1769074766.363997:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074766.364015:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.364042:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.364057:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446800, rc 0) 00000100:00100000:0.0:1769074766.364118:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62721:x1855007616773888:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4194us (4408us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074766.364128:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13915 00000100:00100000:2.0:1769074766.372500:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.372547:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616774016 00000100:00100000:0.0:1769074766.372557:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616774016 00002000:00100000:0.0:1769074766.372600:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2609152->2613247) req@ffff89b5475dbb80 x1855007616774016/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074766.372613:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475dbb80 with x1855007616774016 ext(2609152->2613247) 00010000:00010000:0.0:1769074766.372622:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:0.0:1769074766.372637:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:0.0:1769074766.372644:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5475dbb80 00000100:00100000:1.0:1769074766.372668:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625108352:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074766.372677:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 724 00000100:00100000:0.0:1769074766.372685:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616774016:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:1.0:1769074766.372687:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.372732:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616773504:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 33961us (34194us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.372740:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6924 00000100:00100000:1.0:1769074766.373086:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616774016:12345-192.168.204.60@tcp:3:cat.0 Request processed in 400us (595us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074766.373094:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 724 00002000:00100000:1.0:1769074766.373100:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2609152->2613247) req@ffff89b5475dbb80 x1855007616774016/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.373108:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475dbb80 with x1855007616774016 ext(2609152->2613247) 00010000:00010000:1.0:1769074766.373113:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.373125:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:1.0:1769074766.373136:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5475dbb80 00000100:00100000:2.0:1769074766.399530:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.399636:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616774144 00000100:00100000:1.0:1769074766.399646:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616774144 00002000:00100000:1.0:1769074766.399691:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2859008->2863103) req@ffff89b5475d8000 x1855007616774144/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.399704:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8000 with x1855007616774144 ext(2859008->2863103) 00010000:00010000:1.0:1769074766.399713:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.399728:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:1.0:1769074766.399736:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5475d8000 00000100:00100000:1.0:1769074766.399768:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 725 00000100:00100000:1.0:1769074766.399785:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616774144:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074766.399988:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616774144:12345-192.168.204.60@tcp:3:cat.0 Request processed in 204us (467us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074766.400004:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 725 00002000:00100000:1.0:1769074766.400011:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2859008->2863103) req@ffff89b5475d8000 x1855007616774144/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.400020:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8000 with x1855007616774144 ext(2859008->2863103) 00010000:00010000:1.0:1769074766.400025:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.400042:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:1.0:1769074766.400050:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5475d8000 00000100:00100000:1.0:1769074766.409757:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.412594:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616774272 00000100:00100000:2.0:1769074766.412613:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616774272 00002000:00100000:3.0:1769074766.412790:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2609152->2863103) req@ffff89b546c6a680 x1855007616774272/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.412802:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6a680 with x1855007616774272 ext(2609152->2863103) 00010000:00010000:3.0:1769074766.412827:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.412843:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074766.412851:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546c6a680 00000100:00100000:3.0:1769074766.412939:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 726 00000100:00100000:3.0:1769074766.412950:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616774272:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074766.433177:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616774272:12345-192.168.204.60@tcp:4:cat.0 Request processed in 20210us (23412us total) trans 21474841181 rc 0/0 00000100:00100000:2.0:1769074766.433196:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 726 00002000:00100000:2.0:1769074766.433214:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2609152->2863103) req@ffff89b546c6a680 x1855007616774272/t21474841181(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.433226:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6a680 with x1855007616774272 ext(2609152->2863103) 00010000:00010000:2.0:1769074766.433240:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 15 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074766.433265:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 15 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074766.433273:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b546c6a680 00000100:00100000:0.0:1769074766.437781:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.440733:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616774528 00000100:00100000:1.0:1769074766.440748:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616774528 00000100:00100000:1.0:1769074766.440880:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13301 00000100:00100000:1.0:1769074766.440891:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+238:62727:x1855007616774528:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074766.440916:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.440939:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53426 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7a7 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.441027:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441042:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441052:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441064:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441070:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441076:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441168:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5343b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441178:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5343b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441184:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5343b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074766.441193:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074766.441274:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441281:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441286:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441293:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441299:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074766.441303:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441310:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441321:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074766.441324:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074766.441328:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441338:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd5342d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441351:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441356:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441361:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441367:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074766.441369:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441374:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441380:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074766.441382:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074766.441384:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441389:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53434 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.441432:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53426 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a7 expref: 241 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.441444:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5343b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a7 expref: 240 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.441456:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074766.441458:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074766.441460:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8e00, rc 0) 00000100:00100000:1.0:1769074766.441537:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+240:62727:x1855007616774528:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 648us (3779us total) trans 8589943909 rc 0/0 00000100:00100000:1.0:1769074766.441544:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13301 00000100:00080000:3.0:1769074766.441783:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074766.450803:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.451420:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616774656 00000100:00100000:1.0:1769074766.451432:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616774656 00010000:00010000:1.0:1769074766.451477:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 15 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.451528:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6926 00000100:00100000:1.0:1769074766.451538:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616774656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.451553:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074766.451571:0:6500:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074761 -> 1769074766 00002000:00010000:1.0:1769074766.451574:0:6500:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074761 -> 1769074766 00010000:00010000:1.0:1769074766.451581:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 15 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.451595:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 15 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.451619:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53411 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2609152->2863103) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b792 expref: 15 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.451641:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.451643:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.451652:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.451662:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.451671:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.451673:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.451677:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.451684:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.451720:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.451735:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625108480:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.451862:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.451871:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625108608:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.451896:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074766.452084:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.452149:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.452171:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775040 00000100:00100000:0.0:1769074766.452176:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775040 00000100:00100000:3.0:1769074766.452262:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616774912 00000100:00100000:3.0:1769074766.452266:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616774912 00000100:00100000:3.0:1769074766.452297:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4649 00000100:00100000:3.0:1769074766.452304:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c6850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62727:x1855007616774912:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074766.452386:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 991 00000100:00100000:0.0:1769074766.452395:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:2408:x1855007616775040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074766.452405:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074766.452430:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5343b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a7 expref: 241 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.452445:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074766.452449:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074766.452479:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:2408:x1855007616775040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 86us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074766.452486:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 991 00000100:00100000:3.0:1769074766.452530:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c6850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+240:62727:x1855007616774912:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 225us (446us total) trans 8589943910 rc 0/0 00000100:00100000:3.0:1769074766.452537:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4649 00000100:00100000:1.0:1769074766.467150:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625108480:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.467168:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625108608:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.467197:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.467281:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616774656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 15742us (16486us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.467292:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6926 00000100:00100000:1.0:1769074766.495185:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.495462:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775168 00000100:00100000:2.0:1769074766.495475:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775168 00002000:00100000:2.0:1769074766.495522:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546c6b100 x1855007616775168/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.495534:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6b100 with x1855007616775168 ext(2097152->4194303) 00002000:00010000:2.0:1769074766.495539:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c6b100 00000100:00100000:2.0:1769074766.495572:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8420 00000100:00100000:2.0:1769074766.495582:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616775168:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074766.519513:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.522401:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775296 00000100:00100000:1.0:1769074766.522430:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775296 00002000:00100000:1.0:1769074766.522504:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b6491ed880 x1855007616775296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.522516:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ed880 with x1855007616775296 ext(2097152->4194303) 00002000:00010000:1.0:1769074766.522524:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491ed880 00000100:00100000:1.0:1769074766.522576:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8421 00000100:00100000:1.0:1769074766.522586:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616775296:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074766.570347:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616775168:12345-192.168.204.60@tcp:3:cat.0 Request processed in 74752us (75157us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074766.570365:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8420 00002000:00100000:2.0:1769074766.570381:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546c6b100 x1855007616775168/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.570393:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c6b100 with x1855007616775168 ext(2097152->4194303) 00002000:00010000:2.0:1769074766.570400:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546c6b100 00000100:00100000:1.0:1769074766.604253:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616775296:12345-192.168.204.60@tcp:3:cat.0 Request processed in 81655us (84738us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074766.604267:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8421 00002000:00100000:1.0:1769074766.604279:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b6491ed880 x1855007616775296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.604289:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ed880 with x1855007616775296 ext(2097152->4194303) 00002000:00010000:1.0:1769074766.604295:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6491ed880 00000100:00100000:1.0:1769074766.658497:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074766.658618:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775424 00000100:00100000:0.0:1769074766.658629:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775424 00000100:00100000:0.0:1769074766.658695:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13916 00000100:00100000:0.0:1769074766.658705:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62730:x1855007616775424:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074766.658721:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.658739:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7ae expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.658783:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 24 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.658792:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.658837:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.658845:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.658847:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.658851:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [258048->643071] (req 258048->643071) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.658882:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.658894:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 1854 lvb_type: 1 00000100:00100000:0.0:1769074766.658907:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625108736:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074766.658990:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 10 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.659009:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 10 pid: 27844 timeout: 1854 lvb_type: 1 00000100:00100000:0.0:1769074766.659017:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625108864:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074766.659045:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 24 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.659058:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 24 pid: 8396 timeout: 1854 lvb_type: 0 00000100:00100000:0.0:1769074766.659067:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625108992:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074766.659088:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074766.670536:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625108736:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.670555:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625108864:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.670563:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625108992:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.670601:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [258048->643071] (req 258048->643071) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.670623:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00000100:00100000:1.0:1769074766.670689:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62730:x1855007616775424:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11985us (12208us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.670698:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13916 00000100:00100000:3.0:1769074766.674764:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.674849:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.675202:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775552 00000100:00100000:1.0:1769074766.675215:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775552 00010000:00010000:1.0:1769074766.675270:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.675314:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775680 00000100:00100000:1.0:1769074766.675318:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775680 00010000:00010000:1.0:1769074766.675331:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 11 pid: 27844 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.675494:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6927 00000100:00100000:1.0:1769074766.675505:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616775552:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.675523:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074766.675560:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.675582:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.675614:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd531ef lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b634 expref: 20 pid: 8396 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.675638:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.675708:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616775552:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 202us (963us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.675717:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6927 00000100:00100000:1.0:1769074766.675779:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6928 00000100:00100000:1.0:1769074766.675789:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616775680:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.675800:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074766.675872:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 11 pid: 27844 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.675885:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 11 pid: 27844 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.675902:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb400/0x7fe2323adfd531f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b63b expref: 11 pid: 27844 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.675915:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.675947:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616775680:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 158us (1095us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.675956:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6928 00000100:00100000:2.0:1769074766.688589:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.688672:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775808 00000100:00100000:1.0:1769074766.688682:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775808 00010000:00010000:1.0:1769074766.688720:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 25 pid: 8396 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.688772:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6929 00000100:00100000:1.0:1769074766.688781:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616775808:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.689132:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074766.689160:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 25 pid: 8396 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.689179:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 25 pid: 8396 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.689203:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd531a2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b618 expref: 25 pid: 8396 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.689217:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.689219:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.689235:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.689245:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.689279:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.689291:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625109120:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.689435:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074766.698268:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.698569:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616775936 00000100:00100000:3.0:1769074766.698583:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616775936 00002000:00100000:3.0:1769074766.698643:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (258048->262143) req@ffff89b5475db100 x1855007616775936/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.698657:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475db100 with x1855007616775936 ext(258048->262143) 00002000:00010000:3.0:1769074766.698663:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475db100 00000100:00100000:3.0:1769074766.698712:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8422 00000100:00100000:3.0:1769074766.698727:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616775936:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074766.698932:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616775936:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 210us (677us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074766.698940:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8422 00002000:00100000:3.0:1769074766.698946:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (258048->262143) req@ffff89b5475db100 x1855007616775936/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.698954:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475db100 with x1855007616775936 ext(258048->262143) 00002000:00010000:3.0:1769074766.698957:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475db100 00000100:00100000:1.0:1769074766.700903:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625109120:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074766.700937:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.701012:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616775808:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12231us (12436us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.701023:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6929 00000100:00100000:2.0:1769074766.715727:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.716241:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776064 00000100:00100000:1.0:1769074766.716253:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776064 00002000:00100000:1.0:1769074766.716314:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (638976->643071) req@ffff89b5475da680 x1855007616776064/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.716327:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475da680 with x1855007616776064 ext(638976->643071) 00002000:00010000:1.0:1769074766.716334:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475da680 00000100:00100000:1.0:1769074766.716460:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8423 00000100:00100000:1.0:1769074766.716475:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616776064:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074766.716734:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616776064:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 260us (1013us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074766.716744:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8423 00002000:00100000:1.0:1769074766.716771:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (638976->643071) req@ffff89b5475da680 x1855007616776064/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.716782:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475da680 with x1855007616776064 ext(638976->643071) 00002000:00010000:1.0:1769074766.716786:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475da680 00000100:00100000:0.0:1769074766.751369:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.751575:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776192 00000100:00100000:1.0:1769074766.751586:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776192 00000100:00100000:1.0:1769074766.751644:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13917 00000100:00100000:1.0:1769074766.751655:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62727:x1855007616776192:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074766.751674:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.751691:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7b5 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.751719:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.751729:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.751737:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 9 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.751744:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.751746:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.751750:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2785280->3145727] (req 2785280->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 25 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.751796:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 9 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.751844:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 9 pid: 8398 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.751858:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625109248:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.752096:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.752110:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 18 pid: 9048 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.752119:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625109376:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.752152:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.752160:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.752169:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625109504:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.752189:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074766.754634:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625109248:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.754649:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625109376:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.754657:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625109504:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.754689:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2785280->3145727] (req 2785280->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 25 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.754707:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:1.0:1769074766.754773:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62727:x1855007616776192:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 3119us (3415us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.754782:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13917 00000100:00100000:3.0:1769074766.766847:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.767397:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776320 00000100:00100000:1.0:1769074766.767416:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776320 00010000:00010000:1.0:1769074766.767462:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 10 pid: 8398 timeout: 1854 lvb_type: 1 00000100:00100000:1.0:1769074766.767505:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6930 00000100:00100000:1.0:1769074766.767515:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616776320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.767529:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074766.767552:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 10 pid: 8398 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.767567:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 10 pid: 8398 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.767591:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd532f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6dc expref: 10 pid: 8398 timeout: 1854 lvb_type: 1 00010000:00010000:1.0:1769074766.767612:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074766.767674:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616776320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 161us (879us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.767682:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6930 00000100:00100000:0.0:1769074766.770844:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.771110:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776448 00000100:00100000:3.0:1769074766.771124:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776448 00010000:00010000:3.0:1769074766.771166:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074766.771213:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6931 00000100:00100000:3.0:1769074766.771222:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616776448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.771249:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074766.771272:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.771285:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.771312:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5326d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b681 expref: 25 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.771333:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.771662:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616776448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 438us (858us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.771671:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6931 00000100:00100000:0.0:1769074766.774298:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.774369:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776576 00000100:00100000:3.0:1769074766.774378:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776576 00010000:00010000:3.0:1769074766.774426:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 19 pid: 9048 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074766.774474:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6932 00000100:00100000:3.0:1769074766.774483:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616776576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.774497:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074766.774517:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 19 pid: 9048 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.774532:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 19 pid: 9048 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.774556:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd53323 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b6f8 expref: 19 pid: 9048 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.774574:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074766.774576:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074766.774584:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.774593:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.774628:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074766.774642:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625109632:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074766.774721:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b647316240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.777245:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625109632:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074766.777269:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.777330:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616776576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 2846us (3040us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.777340:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6932 00000100:00100000:0.0:1769074766.780903:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.781009:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776704 00000100:00100000:1.0:1769074766.781022:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776704 00002000:00100000:1.0:1769074766.781088:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2785280->2789375) req@ffff89b6491efb80 x1855007616776704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.781101:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491efb80 with x1855007616776704 ext(2785280->2789375) 00002000:00010000:1.0:1769074766.781108:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491efb80 00000100:00100000:1.0:1769074766.781142:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8424 00000100:00100000:1.0:1769074766.781152:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616776704:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074766.781353:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616776704:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 202us (466us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074766.781361:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8424 00002000:00100000:1.0:1769074766.781368:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2785280->2789375) req@ffff89b6491efb80 x1855007616776704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074766.781376:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491efb80 with x1855007616776704 ext(2785280->2789375) 00002000:00010000:1.0:1769074766.781380:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491efb80 00000100:00100000:0.0:1769074766.805799:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.805970:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776832 00000100:00100000:1.0:1769074766.805986:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776832 00000100:00100000:1.0:1769074766.806065:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13918 00000100:00100000:1.0:1769074766.806078:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62727:x1855007616776832:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074766.806107:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.806132:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7bc expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.806176:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 19 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.806186:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.806193:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.806195:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.806199:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->2646015] (req 2097152->2646015) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.806240:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.806255:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.806270:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625109760:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.806382:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 19 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.806392:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 19 pid: 8396 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.806401:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625109888:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.806437:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074766.818994:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.819021:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.819057:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.819285:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777216 00000100:00100000:3.0:1769074766.819297:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777216 00000100:00100000:3.0:1769074766.819359:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777344 00000100:00100000:3.0:1769074766.819362:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777344 00000100:00100000:3.0:1769074766.819386:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13919 00000100:00100000:3.0:1769074766.819396:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616777216:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074766.819426:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.819456:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd5345e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7c3 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074766.819526:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625110016:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074766.819606:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b647316240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.819626:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777088 00000100:00100000:3.0:1769074766.819630:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777088 00010000:00010000:3.0:1769074766.819654:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 20 pid: 8396 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074766.819681:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6933 00000100:00100000:3.0:1769074766.819688:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616777088:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.819700:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074766.819721:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 20 pid: 8396 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.819732:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 20 pid: 8396 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.819754:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.819778:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616777088:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 90us (799us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.819785:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6933 00000100:00100000:0.0:1769074766.819994:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13920 00000100:00100000:0.0:1769074766.820004:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616777344:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.820011:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.820020:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53465 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7ca expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074766.820033:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074766.820040:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53465 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ca expref: 15 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.820054:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53465 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ca expref: 15 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.820065:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45600, rc 0) 00000100:00100000:0.0:1769074766.820087:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616777344:12345-192.168.204.60@tcp:101:cat.0 Request processed in 85us (1029us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074766.820093:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13920 00002000:00010000:3.0:1769074766.822742:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074764 -> 1769074766 00002000:00010000:3.0:1769074766.822751:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074764 -> 1769074766 00002000:00010000:3.0:1769074766.822753:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074764 -> 1769074766 00000100:00100000:3.0:1769074766.822782:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625110016:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074766.822867:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd5345e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7c3 expref: 19 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074766.822889:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd5345e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7c3 expref: 19 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074766.822902:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70c00, rc 0) 00000100:00100000:3.0:1769074766.822964:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616777216:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3569us (3941us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074766.822973:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13919 00000100:00100000:1.0:1769074766.823991:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625109760:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.824011:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53418 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b799 expref: 19 pid: 8396 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.824027:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625109888:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.824053:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->2646015] (req 2097152->2646015) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.824073:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9400, rc 0) 00000100:00100000:1.0:1769074766.824122:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62727:x1855007616776832:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 18047us (18340us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.824130:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13918 00000100:00100000:0.0:1769074766.825169:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.826960:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616776960 00000100:00100000:3.0:1769074766.826982:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616776960 00010000:00010000:3.0:1769074766.827040:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074766.827086:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6934 00000100:00100000:3.0:1769074766.827097:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616776960:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.827120:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074766.827145:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.827159:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.827185:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd5341f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7a0 expref: 14 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.827212:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074766.827215:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074766.827223:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.827235:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.827279:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074766.827295:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625110144:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074766.827399:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b647316240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.836139:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625110144:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074766.836178:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.836245:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616776960:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9148us (11082us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.836261:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6934 00000100:00100000:2.0:1769074766.843529:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.843575:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.843662:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777472 00000100:00100000:3.0:1769074766.843674:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777472 00002000:00100000:3.0:1769074766.843723:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5475d8380 x1855007616777472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 504/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.843735:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8380 with x1855007616777472 ext(0->1048575) 00002000:00010000:3.0:1769074766.843741:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475d8380 00000100:00100000:3.0:1769074766.843775:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777600 00000100:00100000:3.0:1769074766.843779:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777600 00002000:00100000:3.0:1769074766.843797:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2641920->2646015) req@ffff89b5475dad80 x1855007616777600/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.843847:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475dad80 with x1855007616777600 ext(2641920->2646015) 00002000:00010000:3.0:1769074766.843851:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475dad80 00000100:00100000:3.0:1769074766.843872:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8425 00000100:00100000:3.0:1769074766.843882:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616777472:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074766.844429:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8426 00000100:00100000:0.0:1769074766.844441:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616777600:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074766.844535:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616777600:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 97us (959us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074766.844543:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8426 00002000:00100000:0.0:1769074766.844549:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2641920->2646015) req@ffff89b5475dad80 x1855007616777600/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/4536 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074766.844558:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475dad80 with x1855007616777600 ext(2641920->2646015) 00002000:00010000:0.0:1769074766.844563:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475dad80 00000100:00100000:3.0:1769074766.864146:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616777472:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20254us (20615us total) trans 0 rc 663552/663552 00000100:00100000:3.0:1769074766.864161:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8425 00002000:00100000:3.0:1769074766.864174:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5475d8380 x1855007616777472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 504/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 663552/663552 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.864185:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8380 with x1855007616777472 ext(0->1048575) 00002000:00010000:3.0:1769074766.864191:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475d8380 00000100:00100000:0.0:1769074766.900645:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.900906:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777856 00000100:00100000:1.0:1769074766.900919:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777856 00000100:00100000:1.0:1769074766.900993:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13302 00000100:00100000:1.0:1769074766.901004:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+240:62740:x1855007616777856:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074766.901033:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.901057:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd5346c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7d1 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.901136:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901152:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901163:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901177:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901184:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901190:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901325:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53481 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901336:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53481 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901343:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53481 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074766.901353:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074766.901444:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901452:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901458:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901463:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901469:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074766.901473:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901479:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901490:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074766.901493:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074766.901498:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901508:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53473 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901523:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901530:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901535:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901540:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074766.901543:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901549:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901556:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074766.901559:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074766.901561:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901567:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd5347a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074766.901609:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd5346c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7d1 expref: 243 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.901622:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53481 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7d1 expref: 242 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.901634:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074766.901637:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074766.901639:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:1.0:1769074766.901725:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:62740:x1855007616777856:12345-192.168.204.60@tcp:101:cat.0 Request processed in 722us (1091us total) trans 8589943911 rc 0/0 00000100:00100000:1.0:1769074766.901734:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13302 00000100:00100000:2.0:1769074766.916968:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.917016:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.917051:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.917230:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616777984 00000100:00100000:1.0:1769074766.917245:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616777984 00000100:00100000:1.0:1769074766.917308:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616778368 00000100:00100000:1.0:1769074766.917311:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616778368 00000100:00100000:1.0:1769074766.917339:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13921 00000100:00100000:1.0:1769074766.917349:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616777984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074766.917370:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.917392:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5348f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7d8 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.917477:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625110272:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074766.917567:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074766.917954:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13922 00000100:00100000:0.0:1769074766.917966:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616778368:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.917977:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.917987:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53496 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7df expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074766.918018:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625110400:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074766.918054:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.918922:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616778240 00000100:00100000:3.0:1769074766.918933:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616778240 00000100:00100000:3.0:1769074766.918998:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4650 00000100:00100000:3.0:1769074766.919007:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:62740:x1855007616778240:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074766.919304:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62740:x1855007616778240:12345-192.168.204.60@tcp:35:cat.0 Request processed in 297us (2348us total) trans 8589943912 rc 0/0 00000100:00100000:3.0:1769074766.919312:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4650 00002000:00010000:0.0:1769074766.926575:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074761 -> 1769074766 00000100:00100000:0.0:1769074766.926612:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625110400:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074766.926648:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53496 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7df expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.926672:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53496 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7df expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.926684:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:0.0:1769074766.926773:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616778368:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8810us (9720us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074766.926782:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13922 00000100:00100000:1.0:1769074766.927300:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625110272:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074766.927322:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5348f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7d8 expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.927336:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5348f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7d8 expref: 19 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074766.927350:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8800, rc 0) 00000100:00100000:1.0:1769074766.927377:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616777984:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10030us (10361us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074766.927383:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13921 00000100:00100000:1.0:1769074766.931836:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.933935:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616778496 00000100:00100000:3.0:1769074766.933952:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616778496 00000100:00100000:3.0:1769074766.934037:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 992 00000100:00100000:3.0:1769074766.934050:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:2407:x1855007616778496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074766.934070:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074766.934103:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53481 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7d1 expref: 243 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.934126:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074766.934132:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074766.934205:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:2407:x1855007616778496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 158us (2396us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.934213:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 992 00000100:00100000:2.0:1769074766.949946:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.950067:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.950165:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616778624 00000100:00100000:1.0:1769074766.950176:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616778624 00000100:00100000:1.0:1769074766.950230:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616778752 00000100:00100000:1.0:1769074766.950233:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616778752 00000100:00100000:0.0:1769074766.950240:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13923 00000100:00100000:0.0:1769074766.950252:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616778624:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:1.0:1769074766.950265:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13924 00000100:00100000:1.0:1769074766.950271:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62736:x1855007616778752:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074766.950274:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074766.950278:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074766.950292:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd534a4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7e6 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.950292:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7ed expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.950323:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 25 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.950331:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074766.950332:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074766.950336:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074766.950351:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625110528:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074766.950356:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 25 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.950372:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 25 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.950380:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625110656:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074766.950540:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074766.954191:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074766.959243:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.959365:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625110656:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074766.959412:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 19 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.959434:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8800, rc 0) 00000100:00100000:1.0:1769074766.959508:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62736:x1855007616778752:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9236us (9441us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074766.959519:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13924 00002000:00010000:0.0:1769074766.959679:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074763 -> 1769074766 00002000:00010000:0.0:1769074766.959685:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074764 -> 1769074766 00002000:00010000:0.0:1769074766.959687:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074763 -> 1769074766 00000100:00100000:0.0:1769074766.959707:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625110528:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074766.959733:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd534a4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7e6 expref: 25 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.959896:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd534a4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7e6 expref: 25 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074766.959918:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:0.0:1769074766.959943:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616778624:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9694us (10012us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074766.959950:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13923 00000100:00100000:3.0:1769074766.960043:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616778880 00000100:00100000:3.0:1769074766.960054:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616778880 00002000:00100000:3.0:1769074766.960102:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2785280->3145727) req@ffff89b5497a8380 x1855007616778880/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.960113:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8380 with x1855007616778880 ext(2785280->3145727) 00010000:00010000:3.0:1769074766.960120:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 24 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.960134:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 24 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074766.960141:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a8380 00000100:00100000:3.0:1769074766.960175:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 727 00000100:00100000:3.0:1769074766.960185:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616778880:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074766.972680:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616778880:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 12481us (13430us total) trans 17179874494 rc 0/0 00000100:00100000:3.0:1769074766.972699:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 727 00002000:00100000:3.0:1769074766.972715:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2785280->3145727) req@ffff89b5497a8380 x1855007616778880/t17179874494(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.972726:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8380 with x1855007616778880 ext(2785280->3145727) 00010000:00010000:3.0:1769074766.972738:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 26 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.972764:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 26 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074766.972774:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a8380 00000100:00100000:0.0:1769074766.972961:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074766.973139:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779008 00000100:00100000:3.0:1769074766.973149:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779008 00000100:00100000:3.0:1769074766.973318:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13925 00000100:00100000:3.0:1769074766.973328:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62740:x1855007616779008:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074766.973348:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074766.973372:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7f4 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.973402:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.973422:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074766.973424:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074766.973429:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.973478:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 18 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.973492:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 18 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074766.973509:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625110784:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074766.973603:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000001:00080000:0.0:1769074766.975128:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874493 is committed 00000001:00080000:0.0:1769074766.975148:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874492 is committed 00000001:00080000:0.0:1769074766.975169:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874487 is committed 00000100:00100000:3.0:1769074766.982723:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625110784:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074766.982773:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074766.982792:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70c00, rc 0) 00000100:00100000:3.0:1769074766.982887:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62740:x1855007616779008:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9561us (9926us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074766.982897:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13925 00000100:00100000:2.0:1769074766.984030:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.984068:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074766.984151:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779264 00000100:00100000:1.0:1769074766.984160:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779264 00010000:00010000:1.0:1769074766.984196:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074766.984235:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6935 00000100:00100000:1.0:1769074766.984245:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616779264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074766.984259:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074766.984282:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.984301:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.984324:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53450 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2785280->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7b5 expref: 23 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074766.984339:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.984342:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074766.984349:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.984361:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074766.984395:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 8 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074766.984422:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625110912:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074766.984507:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074766.986319:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779136 00000100:00100000:3.0:1769074766.986331:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779136 00002000:00100000:3.0:1769074766.986378:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (258048->643071) req@ffff89b5497a9c00 x1855007616779136/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074766.986391:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9c00 with x1855007616779136 ext(258048->643071) 00010000:00010000:3.0:1769074766.986401:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 9 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074766.986433:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 9 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074766.986449:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a9c00 00000100:00100000:3.0:1769074766.986479:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 728 00000100:00100000:3.0:1769074766.986488:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616779136:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074766.997181:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616779136:12345-192.168.204.60@tcp:4:cat.0 Request processed in 10678us (13141us total) trans 17179874495 rc 0/0 00000100:00100000:2.0:1769074766.997197:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 728 00002000:00100000:2.0:1769074766.997211:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (258048->643071) req@ffff89b5497a9c00 x1855007616779136/t17179874495(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/448 e 0 to 0 dl 1769074777 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.997231:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9c00 with x1855007616779136 ext(258048->643071) 00010000:00010000:2.0:1769074766.997242:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074766.997261:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074766.997270:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497a9c00 00000100:00100000:3.0:1769074766.997998:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074766.998884:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779392 00000100:00100000:2.0:1769074766.998895:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779392 00002000:00100000:2.0:1769074766.998940:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54d102a00 x1855007616779392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/0 e 0 to 0 dl 1769074777 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074766.998950:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d102a00 with x1855007616779392 ext(2097152->3145727) 00002000:00010000:2.0:1769074766.998957:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d102a00 00000100:00100000:2.0:1769074766.999043:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8427 00000100:00100000:2.0:1769074766.999053:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616779392:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074767.002797:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625110912:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074767.002842:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.002912:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616779264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 18668us (18842us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.002923:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6935 00000100:00100000:3.0:1769074767.005957:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.007850:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779520 00000100:00100000:1.0:1769074767.007866:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779520 00010000:00010000:1.0:1769074767.007918:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074767.007958:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6936 00000100:00100000:1.0:1769074767.007968:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616779520:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.007983:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.008007:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.008021:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.008044:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53449 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 258048->643071) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ae expref: 13 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.008061:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.008063:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.008069:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.008079:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.008117:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.008130:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625111040:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074767.008206:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00080000:3.0:1769074767.009144:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074767.013322:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625111040:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074767.013357:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.013435:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616779520:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5467us (7488us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.013453:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6936 00000100:00100000:2.0:1769074767.018392:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616779392:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 19331us (20388us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074767.018415:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8427 00002000:00100000:2.0:1769074767.018428:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54d102a00 x1855007616779392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:302/0 lens 488/440 e 0 to 0 dl 1769074777 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.018438:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d102a00 with x1855007616779392 ext(2097152->3145727) 00002000:00010000:2.0:1769074767.018445:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d102a00 00000100:00100000:2.0:1769074767.022908:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.023094:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779648 00000100:00100000:1.0:1769074767.023105:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779648 00002000:00100000:1.0:1769074767.023169:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5497aa680 x1855007616779648/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.023180:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aa680 with x1855007616779648 ext(0->1048575) 00002000:00010000:1.0:1769074767.023185:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aa680 00000100:00100000:1.0:1769074767.023223:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8428 00000100:00100000:1.0:1769074767.023232:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616779648:12345-192.168.204.60@tcp:3:cat.0 00000100:00080000:2.0:1769074767.025325:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074767.038293:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616779648:12345-192.168.204.60@tcp:3:cat.0 Request processed in 15051us (15385us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074767.038308:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8428 00002000:00100000:1.0:1769074767.038322:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5497aa680 x1855007616779648/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.038332:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aa680 with x1855007616779648 ext(0->1048575) 00002000:00010000:1.0:1769074767.038338:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aa680 00000100:00100000:2.0:1769074767.051638:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.051690:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.051764:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779776 00000100:00100000:1.0:1769074767.051781:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779776 00000100:00100000:1.0:1769074767.051868:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616779904 00000100:00100000:1.0:1769074767.051872:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616779904 00000100:00100000:1.0:1769074767.051895:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13926 00000100:00100000:1.0:1769074767.051906:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62736:x1855007616779776:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074767.051923:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074767.051942:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b7fb expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.051986:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.051996:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074767.051998:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074767.052002:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.052038:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 17 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.052052:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 17 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074767.052067:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625111168:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074767.052155:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074767.052302:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13927 00000100:00100000:0.0:1769074767.052313:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:62742:x1855007616779904:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074767.052321:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074767.052333:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b802 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.052350:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.052357:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.052358:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.052361:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2899968->3108863] (req 2899968->3108863) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.052380:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.052389:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:0.0:1769074767.052399:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625111296:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074767.052441:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00080000:2.0:1769074767.053106:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074767.057411:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625111168:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074767.057467:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.057490:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f200, rc 0) 00000100:00100000:0.0:1769074767.057572:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62736:x1855007616779776:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5666us (5944us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.057583:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13926 00000100:00100000:3.0:1769074767.057966:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625111296:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074767.058042:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2899968->3108863] (req 2899968->3108863) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.058055:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640030a00, rc 0) 00000100:00100000:3.0:1769074767.058089:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62742:x1855007616779904:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 5779us (6398us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.058097:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13927 00000100:00100000:2.0:1769074767.064787:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.064912:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.065015:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780160 00000100:00100000:1.0:1769074767.065029:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780160 00010000:00010000:1.0:1769074767.065094:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074767.065140:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6937 00000100:00100000:1.0:1769074767.065153:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616780160:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.065174:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.065206:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.065219:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.065245:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd5349d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7ed expref: 11 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.065267:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.065269:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.065277:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.065289:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.065332:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.065349:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625111424:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074767.065449:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0000 going to sleep for 11 seconds 00000100:00100000:3.0:1769074767.066164:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780032 00000100:00100000:3.0:1769074767.066172:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780032 00002000:00100000:3.0:1769074767.066214:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2646015) req@ffff89b54cde4a80 x1855007616780032/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.066236:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4a80 with x1855007616780032 ext(2097152->2646015) 00010000:00010000:3.0:1769074767.066245:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 18 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:3.0:1769074767.066264:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 18 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:3.0:1769074767.066272:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cde4a80 00000100:00100000:3.0:1769074767.066303:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 729 00000100:00100000:3.0:1769074767.066311:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616780032:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074767.079881:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.080082:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.080112:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780416 00000100:00100000:0.0:1769074767.080122:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780416 00002000:00100000:0.0:1769074767.080178:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2899968->2904063) req@ffff89b54920b480 x1855007616780416/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.080190:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920b480 with x1855007616780416 ext(2899968->2904063) 00002000:00010000:0.0:1769074767.080196:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54920b480 00000100:00100000:0.0:1769074767.080234:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8429 00000100:00100000:0.0:1769074767.080244:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616780416:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074767.080454:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616780416:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 210us (370us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074767.080462:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8429 00002000:00100000:0.0:1769074767.080470:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2899968->2904063) req@ffff89b54920b480 x1855007616780416/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/4536 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.080478:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920b480 with x1855007616780416 ext(2899968->2904063) 00002000:00010000:0.0:1769074767.080482:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54920b480 00000100:00100000:1.0:1769074767.081497:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625111424:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074767.081518:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.081585:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616780160:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16435us (16807us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.081594:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6937 00000100:00100000:2.0:1769074767.082230:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616780032:12345-192.168.204.60@tcp:4:cat.0 Request processed in 15917us (17314us total) trans 21474841182 rc 0/0 00000100:00100000:2.0:1769074767.082240:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 729 00002000:00100000:2.0:1769074767.082249:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2646015) req@ffff89b54cde4a80 x1855007616780032/t21474841182(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/448 e 0 to 0 dl 1769074778 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.082259:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4a80 with x1855007616780032 ext(2097152->2646015) 00010000:00010000:2.0:1769074767.082269:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 20 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074767.082284:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 20 pid: 27844 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074767.082295:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cde4a80 00000100:00100000:3.0:1769074767.082352:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780288 00000100:00100000:3.0:1769074767.082359:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780288 00000100:00100000:3.0:1769074767.082413:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13928 00000100:00100000:3.0:1769074767.082423:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62740:x1855007616780288:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.082437:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.082452:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b809 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.082475:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.082484:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074767.082486:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074767.082491:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.082520:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.082535:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 9 pid: 8398 timeout: 1854 lvb_type: 0 00000100:00100000:3.0:1769074767.082547:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625111552:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074767.082753:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074767.092526:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.092630:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780544 00000100:00100000:1.0:1769074767.092642:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780544 00010000:00010000:1.0:1769074767.092681:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 20 pid: 27844 timeout: 1854 lvb_type: 0 00000100:00100000:1.0:1769074767.092730:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6938 00000100:00100000:1.0:1769074767.092740:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616780544:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.092771:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.092794:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 20 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.092845:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 20 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.092868:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53457 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2646015) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7bc expref: 20 pid: 27844 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.092883:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.092886:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.092893:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.092902:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.092943:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.092955:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625111680:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074767.093033:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0000 going to sleep for 11 seconds 00000100:00100000:3.0:1769074767.095613:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625111552:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074767.095651:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.095670:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70000, rc 0) 00000100:00100000:3.0:1769074767.095761:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62740:x1855007616780288:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13340us (15895us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.095770:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13928 00000100:00100000:0.0:1769074767.101952:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.102197:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625111680:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074767.102224:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.102284:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616780544:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9544us (9769us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.102297:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6938 00000100:00100000:2.0:1769074767.102453:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780672 00000100:00100000:2.0:1769074767.102463:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780672 00002000:00100000:2.0:1769074767.102511:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3104768->3108863) req@ffff89b549208e00 x1855007616780672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.102522:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549208e00 with x1855007616780672 ext(3104768->3108863) 00010000:00010000:2.0:1769074767.102530:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074767.102545:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074767.102554:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549208e00 00000100:00100000:2.0:1769074767.102583:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 730 00000100:00100000:2.0:1769074767.102593:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616780672:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074767.102743:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616780672:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 151us (805us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074767.102750:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 730 00002000:00100000:2.0:1769074767.102757:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3104768->3108863) req@ffff89b549208e00 x1855007616780672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/4536 e 0 to 0 dl 1769074778 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.102765:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549208e00 with x1855007616780672 ext(3104768->3108863) 00010000:00010000:2.0:1769074767.102770:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074767.102781:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074767.102788:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549208e00 00000100:00100000:0.0:1769074767.131722:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.133300:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780800 00000100:00100000:1.0:1769074767.133319:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780800 00002000:00100000:1.0:1769074767.133382:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2899968->3108863) req@ffff89b549209f80 x1855007616780800/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.133395:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209f80 with x1855007616780800 ext(2899968->3108863) 00010000:00010000:1.0:1769074767.133414:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:1.0:1769074767.133431:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 10 pid: 8398 timeout: 1854 lvb_type: 0 00002000:00010000:1.0:1769074767.133439:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549209f80 00000100:00100000:1.0:1769074767.133489:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 731 00000100:00100000:1.0:1769074767.133500:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616780800:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074767.140214:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.140451:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616780928 00000100:00100000:2.0:1769074767.140462:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616780928 00002000:00100000:2.0:1769074767.140516:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d100700 x1855007616780928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.140529:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100700 with x1855007616780928 ext(2097152->4194303) 00002000:00010000:2.0:1769074767.140536:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d100700 00000100:00100000:2.0:1769074767.140573:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8430 00000100:00100000:2.0:1769074767.140583:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616780928:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074767.150443:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.150684:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616781056 00000100:00100000:2.0:1769074767.150696:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616781056 00002000:00100000:2.0:1769074767.150747:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b549209c00 x1855007616781056/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.150763:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209c00 with x1855007616781056 ext(3145728->4194303) 00010000:00010000:2.0:1769074767.150771:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:2.0:1769074767.150787:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1854 lvb_type: 0 00002000:00010000:2.0:1769074767.150794:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549209c00 00000100:00100000:2.0:1769074767.150855:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 732 00000100:00100000:2.0:1769074767.150864:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616781056:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074767.165327:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.165448:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616781312 00000100:00100000:2.0:1769074767.165460:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616781312 00000100:00100000:2.0:1769074767.165530:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13303 00000100:00100000:2.0:1769074767.165540:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:62744:x1855007616781312:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074767.165567:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074767.165586:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd534c7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b810 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.165666:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165681:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165696:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165714:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165720:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165726:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165875:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd534dc lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165887:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd534dc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.165894:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd534dc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074767.165903:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074767.165993:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166001:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166006:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166012:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166018:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074767.166023:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166031:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166043:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074767.166045:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074767.166049:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166055:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd534ce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166069:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166075:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166080:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166084:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074767.166087:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166094:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166100:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4000) 00010000:00010000:2.0:1769074767.166103:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074767.166106:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166110:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd534d5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.166159:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd534c7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b810 expref: 245 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.166174:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd534dc lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b810 expref: 244 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.166181:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074767.166183:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074767.166186:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:2.0:1769074767.166264:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:62744:x1855007616781312:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 726us (949us total) trans 8589943913 rc 0/0 00000100:00100000:2.0:1769074767.166273:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13303 00000100:00100000:0.0:1769074767.171496:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616780800:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 37982us (39777us total) trans 17179874496 rc 0/0 00000100:00100000:0.0:1769074767.171516:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 731 00002000:00100000:0.0:1769074767.171532:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2899968->3108863) req@ffff89b549209f80 x1855007616780800/t17179874496(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/448 e 0 to 0 dl 1769074778 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.171544:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209f80 with x1855007616780800 ext(2899968->3108863) 00010000:00010000:0.0:1769074767.171556:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 14 pid: 8398 timeout: 1854 lvb_type: 0 00010000:00010000:0.0:1769074767.171577:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 14 pid: 8398 timeout: 1855 lvb_type: 0 00002000:00010000:0.0:1769074767.171586:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549209f80 00000100:00100000:2.0:1769074767.180190:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616780928:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 39595us (39974us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074767.180206:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8430 00002000:00100000:2.0:1769074767.180231:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d100700 x1855007616780928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.180243:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100700 with x1855007616780928 ext(2097152->4194303) 00002000:00010000:2.0:1769074767.180249:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d100700 00000100:00100000:2.0:1769074767.183793:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616781056:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 32917us (33351us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074767.183846:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 732 00002000:00100000:2.0:1769074767.183860:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b549209c00 x1855007616781056/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.183877:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209c00 with x1855007616781056 ext(3145728->4194303) 00010000:00010000:2.0:1769074767.183885:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:2.0:1769074767.183902:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00002000:00010000:2.0:1769074767.183908:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549209c00 00000100:00100000:0.0:1769074767.192227:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.192273:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.192299:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.192482:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616781696 00000100:00100000:1.0:1769074767.192494:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616781696 00000100:00100000:1.0:1769074767.192564:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616781824 00000100:00100000:1.0:1769074767.192568:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616781824 00010000:00010000:1.0:1769074767.192586:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.192621:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6939 00000100:00100000:1.0:1769074767.192633:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616781824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.192652:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.192678:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.192693:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.192723:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b640030a00/0x7fe2323adfd534b9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2899968->3108863) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b802 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.192741:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.192754:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.192764:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.192776:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.192859:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.192876:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625111808:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074767.192950:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0000 going to sleep for 11 seconds 00000100:00100000:2.0:1769074767.193203:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 993 00000100:00100000:2.0:1769074767.193226:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:2407:x1855007616781696:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074767.193238:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074767.193256:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd534dc lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b810 expref: 244 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.193270:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074767.193274:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074767.193305:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:2407:x1855007616781696:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 93us (1031us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074767.193312:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 993 00000100:00100000:3.0:1769074767.193595:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616781568 00000100:00100000:3.0:1769074767.193605:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616781568 00000100:00100000:3.0:1769074767.193685:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4651 00000100:00100000:3.0:1769074767.193694:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62744:x1855007616781568:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074767.193968:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:62744:x1855007616781568:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 276us (1765us total) trans 8589943914 rc 0/0 00000100:00100000:3.0:1769074767.193976:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4651 00000100:00100000:0.0:1769074767.206390:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.206390:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625111808:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074767.206425:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.206494:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616781824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13861us (14191us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.206505:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6939 00000100:00100000:3.0:1769074767.207080:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616781952 00000100:00100000:3.0:1769074767.207089:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616781952 00000100:00100000:3.0:1769074767.207171:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13929 00000100:00100000:3.0:1769074767.207182:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62736:x1855007616781952:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.207200:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.207229:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b817 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.207256:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.207259:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.207269:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.207285:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.207297:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:3.0:1769074767.207355:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62736:x1855007616781952:12345-192.168.204.60@tcp:101:cat.0 Request processed in 175us (973us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.207361:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13929 00000100:00100000:2.0:1769074767.212324:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.212430:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782080 00000100:00100000:3.0:1769074767.212439:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782080 00002000:00100000:3.0:1769074767.212499:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54cde6300 x1855007616782080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.212511:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde6300 with x1855007616782080 ext(2097152->3145727) 00002000:00010000:3.0:1769074767.212516:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde6300 00000100:00100000:3.0:1769074767.212656:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8431 00000100:00100000:3.0:1769074767.212667:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616782080:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074767.223320:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.223427:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782208 00000100:00100000:2.0:1769074767.223440:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782208 00002000:00100000:2.0:1769074767.223508:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54920a680 x1855007616782208/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.223521:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920a680 with x1855007616782208 ext(3145728->4194303) 00002000:00010000:2.0:1769074767.223530:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54920a680 00000100:00100000:2.0:1769074767.223579:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8432 00000100:00100000:2.0:1769074767.223591:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616782208:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074767.237752:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616782080:12345-192.168.204.60@tcp:3:cat.0 Request processed in 25074us (25425us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074767.237767:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8431 00002000:00100000:3.0:1769074767.237781:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54cde6300 x1855007616782080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.237791:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde6300 with x1855007616782080 ext(2097152->3145727) 00002000:00010000:3.0:1769074767.237797:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde6300 00000100:00100000:3.0:1769074767.245917:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616782208:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 22315us (22600us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074767.245933:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8432 00002000:00100000:3.0:1769074767.245947:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54920a680 x1855007616782208/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.245958:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920a680 with x1855007616782208 ext(3145728->4194303) 00002000:00010000:3.0:1769074767.245965:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54920a680 00000100:00100000:0.0:1769074767.255385:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.255550:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782336 00000100:00100000:3.0:1769074767.255571:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782336 00000100:00100000:3.0:1769074767.255629:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13930 00000100:00100000:3.0:1769074767.255639:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:62740:x1855007616782336:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.255655:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.255675:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b81e expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.255702:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.255705:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.255713:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 20 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.255732:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 20 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.255756:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:3.0:1769074767.255843:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62740:x1855007616782336:12345-192.168.204.60@tcp:101:cat.0 Request processed in 204us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.255850:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13930 00010000:00010000:2.0:1769074767.261506:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074767.261745:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.262099:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782464 00000100:00100000:0.0:1769074767.262112:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782464 00000100:00100000:0.0:1769074767.262195:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13931 00000100:00100000:0.0:1769074767.262210:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616782464:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074767.262232:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074767.262257:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b825 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.262286:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.262289:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.262302:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074767.262325:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074767.262345:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031400, rc 0) 00000100:00100000:0.0:1769074767.262443:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616782464:12345-192.168.204.60@tcp:101:cat.0 Request processed in 235us (700us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.262450:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13931 00000100:00100000:0.0:1769074767.288153:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.288231:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782592 00000100:00100000:3.0:1769074767.288243:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782592 00002000:00100000:3.0:1769074767.288301:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54920b100 x1855007616782592/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.288313:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920b100 with x1855007616782592 ext(4194304->4808703) 00002000:00010000:3.0:1769074767.288319:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54920b100 00000100:00100000:3.0:1769074767.288356:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8433 00000100:00100000:3.0:1769074767.288366:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616782592:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074767.296298:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616782592:12345-192.168.204.60@tcp:3:cat.0 Request processed in 7922us (8144us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074767.296312:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8433 00002000:00100000:3.0:1769074767.296326:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54920b100 x1855007616782592/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.296337:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920b100 with x1855007616782592 ext(4194304->4808703) 00002000:00010000:3.0:1769074767.296343:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54920b100 00000100:00100000:2.0:1769074767.306618:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.306681:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.306904:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782720 00000100:00100000:0.0:1769074767.306916:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782720 00000100:00100000:3.0:1769074767.306926:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782848 00000100:00100000:3.0:1769074767.306930:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782848 00002000:00100000:3.0:1769074767.306968:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54cde6680 x1855007616782848/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.306979:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde6680 with x1855007616782848 ext(3145728->4194303) 00002000:00100000:0.0:1769074767.306980:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54cde5c00 x1855007616782720/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00010000:3.0:1769074767.306986:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde6680 00002000:00100000:0.0:1769074767.306987:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde5c00 with x1855007616782720 ext(2097152->4194303) 00002000:00010000:0.0:1769074767.306992:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde5c00 00000100:00100000:3.0:1769074767.307022:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8434 00000100:00100000:0.0:1769074767.307027:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8435 00000100:00100000:3.0:1769074767.307031:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616782848:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074767.307033:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616782720:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074767.336483:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616782848:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29397us (29746us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074767.336499:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8434 00002000:00100000:3.0:1769074767.336512:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54cde6680 x1855007616782848/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.336523:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde6680 with x1855007616782848 ext(3145728->4194303) 00002000:00010000:3.0:1769074767.336532:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde6680 00000100:00100000:1.0:1769074767.349570:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616782720:12345-192.168.204.60@tcp:3:cat.0 Request processed in 42521us (42948us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074767.349585:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8435 00002000:00100000:1.0:1769074767.349598:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54cde5c00 x1855007616782720/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.349608:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde5c00 with x1855007616782720 ext(2097152->4194303) 00002000:00010000:1.0:1769074767.349615:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde5c00 00000100:00100000:2.0:1769074767.391128:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.391208:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616782976 00000100:00100000:1.0:1769074767.391222:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616782976 00002000:00100000:1.0:1769074767.391299:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54cde4e00 x1855007616782976/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.391313:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4e00 with x1855007616782976 ext(4194304->4808703) 00002000:00010000:1.0:1769074767.391319:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde4e00 00000100:00100000:1.0:1769074767.391364:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8436 00000100:00100000:1.0:1769074767.391375:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616782976:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074767.400784:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.401151:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783104 00000100:00100000:3.0:1769074767.401160:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783104 00000100:00100000:1.0:1769074767.401189:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616782976:12345-192.168.204.60@tcp:3:cat.0 Request processed in 9814us (10070us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074767.401199:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8436 00000100:00100000:3.0:1769074767.401203:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13932 00002000:00100000:1.0:1769074767.401211:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54cde4e00 x1855007616782976/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1769074767.401211:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62744:x1855007616783104:12345-192.168.204.60@tcp:101:multiop.0 00002000:00100000:1.0:1769074767.401221:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4e00 with x1855007616782976 ext(4194304->4808703) 00002000:00010000:1.0:1769074767.401226:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde4e00 00010000:00010000:3.0:1769074767.401226:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.401242:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b82c expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.401272:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.401286:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.401291:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.401294:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.401298:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3784704->4194303] (req 3784704->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.401330:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.401343:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:3.0:1769074767.401357:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625111936:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074767.401427:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.401436:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:3.0:1769074767.401445:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625112064:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074767.401468:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074767.405156:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625111936:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074767.405176:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625112064:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074767.405208:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3784704->4194303] (req 3784704->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.405227:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:3.0:1769074767.405295:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62744:x1855007616783104:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 4084us (4521us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.405305:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13932 00000100:00100000:2.0:1769074767.415498:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.415872:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783232 00000100:00100000:3.0:1769074767.416152:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783232 00000100:00100000:3.0:1769074767.416324:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13933 00000100:00100000:3.0:1769074767.416338:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616783232:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.416365:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.416391:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53506 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b833 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.416438:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.416455:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53506 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b833 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.416482:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53506 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b833 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.416498:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074767.417188:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616783232:12345-192.168.204.60@tcp:101:cat.0 Request processed in 850us (1697us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074767.417203:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13933 00000100:00100000:2.0:1769074767.430683:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.430803:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.433921:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.434218:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783488 00000100:00100000:3.0:1769074767.434235:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783488 00000100:00100000:3.0:1769074767.434342:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13934 00000100:00100000:3.0:1769074767.434355:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616783488:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.434383:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.434420:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5350d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b83a expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.434463:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.434482:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5350d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b83a expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.434521:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd5350d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b83a expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.434536:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074767.434621:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616783488:12345-192.168.204.60@tcp:101:cat.0 Request processed in 268us (715us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074767.434629:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13934 00000100:00100000:0.0:1769074767.435584:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783360 00000100:00100000:0.0:1769074767.435597:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783360 00010000:00010000:0.0:1769074767.435640:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:0.0:1769074767.435680:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783616 00000100:00100000:0.0:1769074767.435685:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783616 00010000:00010000:0.0:1769074767.435711:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 14 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:0.0:1769074767.435741:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6940 00000100:00100000:0.0:1769074767.435751:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616783360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074767.435770:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074767.435828:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.435844:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.435872:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910da00/0x7fe2323adfd534f1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b81e expref: 21 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:2.0:1769074767.436024:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6941 00000100:00100000:2.0:1769074767.436038:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616783616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074767.436048:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074767.436059:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 14 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:2.0:1769074767.436081:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 14 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:2.0:1769074767.436099:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f200/0x7fe2323adfd534b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7fb expref: 14 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:2.0:1769074767.436116:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074767.436119:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074767.436128:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.436140:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.436199:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074767.436220:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625112192:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074767.436334:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00010000:00010000:0.0:1769074767.437184:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074767.437235:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616783360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1482us (6565us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.437246:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6940 00000100:00100000:2.0:1769074767.456930:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.458030:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625112192:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074767.458079:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074767.458163:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616783616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 22128us (27358us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.458179:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6941 00000100:00100000:1.0:1769074767.458599:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783744 00000100:00100000:1.0:1769074767.458612:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783744 00002000:00100000:1.0:1769074767.458678:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3784704->3788799) req@ffff89b54cde7b80 x1855007616783744/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.458691:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7b80 with x1855007616783744 ext(3784704->3788799) 00002000:00010000:1.0:1769074767.458698:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde7b80 00000100:00100000:1.0:1769074767.458733:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8437 00000100:00100000:1.0:1769074767.458754:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616783744:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074767.459021:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616783744:12345-192.168.204.60@tcp:3:cat.0 Request processed in 269us (2106us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074767.459029:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8437 00002000:00100000:1.0:1769074767.459037:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3784704->3788799) req@ffff89b54cde7b80 x1855007616783744/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/4536 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.459047:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7b80 with x1855007616783744 ext(3784704->3788799) 00002000:00010000:1.0:1769074767.459051:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde7b80 00000100:00100000:2.0:1769074767.507167:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.507613:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616783872 00000100:00100000:0.0:1769074767.507629:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616783872 00000100:00100000:0.0:1769074767.507710:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13935 00000100:00100000:0.0:1769074767.507721:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62744:x1855007616783872:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074767.507756:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074767.507782:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b841 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.507870:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.508292:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.508301:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.508303:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.508308:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4431871] (req 4194304->4431871) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 24 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.508505:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.508524:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 12 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.508543:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112320:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074767.508675:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.508686:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.508696:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112448:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074767.508722:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074767.516032:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.516204:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784000 00000100:00100000:0.0:1769074767.516218:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784000 00010000:00010000:0.0:1769074767.516289:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 13 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:0.0:1769074767.516361:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6942 00000100:00100000:0.0:1769074767.516374:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616784000:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074767.516409:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074767.516447:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 13 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.516463:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 13 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.516496:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074767.517050:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd534ea lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b817 expref: 13 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.517084:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112320:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074767.517340:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616784000:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 966us (1603us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.517352:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6942 00000100:00100000:1.0:1769074767.520419:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112448:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074767.520475:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4431871] (req 4194304->4431871) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 24 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.520501:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437400, rc 0) 00000100:00100000:1.0:1769074767.520596:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62744:x1855007616783872:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12875us (13454us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.520607:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13935 00000100:00100000:2.0:1769074767.535056:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.535378:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784128 00000100:00100000:0.0:1769074767.535403:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784128 00010000:00010000:0.0:1769074767.535465:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:0.0:1769074767.535522:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6943 00000100:00100000:0.0:1769074767.535534:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616784128:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074767.535555:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074767.535583:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.535605:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.535636:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd534c0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b809 expref: 24 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.535660:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.535662:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.535672:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.535684:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.535743:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074767.535764:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625112576:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074767.536479:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074767.554670:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.558989:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784256 00000100:00100000:1.0:1769074767.559016:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784256 00002000:00100000:1.0:1769074767.559117:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4427776->4431871) req@ffff89b5476f8380 x1855007616784256/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.559131:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8380 with x1855007616784256 ext(4427776->4431871) 00002000:00010000:1.0:1769074767.559139:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8380 00000100:00100000:1.0:1769074767.559187:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8438 00000100:00100000:1.0:1769074767.559197:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616784256:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074767.559548:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616784256:12345-192.168.204.60@tcp:3:cat.0 Request processed in 351us (4894us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074767.559557:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8438 00002000:00100000:1.0:1769074767.559565:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4427776->4431871) req@ffff89b5476f8380 x1855007616784256/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/4536 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.559575:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8380 with x1855007616784256 ext(4427776->4431871) 00002000:00010000:1.0:1769074767.559578:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8380 00000100:00100000:0.0:1769074767.565995:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625112576:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074767.566042:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074767.571998:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616784128:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 36441us (36935us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.572020:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6943 00000100:00100000:3.0:1769074767.572994:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.577602:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784384 00000100:00100000:1.0:1769074767.577628:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784384 00000100:00100000:1.0:1769074767.577726:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13936 00000100:00100000:1.0:1769074767.577739:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62748:x1855007616784384:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074767.577766:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074767.577794:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b848 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.577903:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.577915:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.577923:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.577926:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.577931:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1097728->1789951] (req 1097728->1789951) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.577982:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.577999:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 22 pid: 28955 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.578016:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112704:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074767.578168:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.578178:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.578188:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112832:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074767.578221:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074767.584086:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112704:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074767.584454:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625112832:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074767.584503:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1097728->1789951] (req 1097728->1789951) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.584534:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:1.0:1769074767.584635:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62748:x1855007616784384:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6897us (11647us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.584647:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13936 00000100:00100000:2.0:1769074767.584788:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.585055:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.598113:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784512 00000100:00100000:0.0:1769074767.598138:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784512 00010000:00010000:1.0:1769074767.598506:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 23 pid: 28955 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.598577:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784640 00000100:00100000:1.0:1769074767.598582:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784640 00010000:00010000:1.0:1769074767.598599:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.598633:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6944 00000100:00100000:1.0:1769074767.598646:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616784512:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.598670:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.598698:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 23 pid: 28955 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.598715:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 23 pid: 28955 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.598742:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd5340a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b78b expref: 23 pid: 28955 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.598765:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.598903:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616784512:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 258us (14116us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.598913:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6944 00000100:00100000:1.0:1769074767.598939:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6945 00000100:00100000:1.0:1769074767.598947:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616784640:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.598957:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.598966:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.598977:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.598992:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53403 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b784 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.599003:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.599006:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.599015:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.599027:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.599090:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.599108:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625112960:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074767.599178:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074767.624800:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.624889:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.625085:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784896 00000100:00100000:0.0:1769074767.625100:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784896 00000100:00100000:0.0:1769074767.625187:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13937 00000100:00100000:0.0:1769074767.625201:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616784896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074767.625231:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074767.625263:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b437e00/0x7fe2323adfd53522 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b84f expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074767.625360:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625113088:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074767.625497:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074767.625683:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616784768 00000100:00100000:2.0:1769074767.625689:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616784768 00002000:00100000:2.0:1769074767.625733:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1097728->1101823) req@ffff89b54d100380 x1855007616784768/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.625744:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100380 with x1855007616784768 ext(1097728->1101823) 00002000:00010000:2.0:1769074767.625751:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d100380 00000100:00100000:2.0:1769074767.625784:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8439 00000100:00100000:2.0:1769074767.625793:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616784768:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074767.626021:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616784768:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 228us (1235us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074767.626032:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8439 00002000:00100000:2.0:1769074767.626041:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1097728->1101823) req@ffff89b54d100380 x1855007616784768/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/4536 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.626049:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100380 with x1855007616784768 ext(1097728->1101823) 00002000:00010000:2.0:1769074767.626054:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d100380 00000100:00100000:1.0:1769074767.626600:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625112960:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074767.626627:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.626676:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616784640:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 27730us (41621us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.626686:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6945 00000100:00100000:2.0:1769074767.628376:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.628540:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616785024 00000100:00100000:3.0:1769074767.628551:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616785024 00000100:00100000:3.0:1769074767.628618:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13938 00000100:00100000:3.0:1769074767.628629:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616785024:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.628653:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.628678:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53529 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b856 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074767.628765:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625113216:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074767.630441:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00002000:00010000:0.0:1769074767.642497:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074766 -> 1769074767 00002000:00010000:0.0:1769074767.642512:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074766 -> 1769074767 00002000:00010000:0.0:1769074767.642515:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074766 -> 1769074767 00000100:00100000:0.0:1769074767.642550:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625113088:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074767.642611:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b437e00/0x7fe2323adfd53522 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b84f expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074767.642651:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437e00/0x7fe2323adfd53522 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b84f expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074767.642672:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437e00, rc 0) 00000100:00100000:0.0:1769074767.642782:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616784896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 17582us (17889us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074767.642795:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13937 00002000:00010000:3.0:1769074767.643312:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074766 -> 1769074767 00002000:00010000:3.0:1769074767.643316:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074766 -> 1769074767 00002000:00010000:3.0:1769074767.643319:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074766 -> 1769074767 00000100:00100000:3.0:1769074767.643340:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625113216:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074767.643374:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53529 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b856 expref: 14 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.643402:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53529 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b856 expref: 14 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.643414:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074767.643450:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616785024:12345-192.168.204.60@tcp:101:cat.0 Request processed in 14823us (15179us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074767.643457:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13938 00000100:00100000:0.0:1769074767.648435:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.648688:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616785152 00000100:00100000:2.0:1769074767.648701:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616785152 00002000:00100000:2.0:1769074767.648769:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1785856->1789951) req@ffff89b549208380 x1855007616785152/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.648784:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549208380 with x1855007616785152 ext(1785856->1789951) 00002000:00010000:2.0:1769074767.648792:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549208380 00000100:00100000:2.0:1769074767.648886:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8440 00000100:00100000:2.0:1769074767.648897:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616785152:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074767.649129:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616785152:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 233us (710us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074767.649138:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8440 00002000:00100000:2.0:1769074767.649147:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1785856->1789951) req@ffff89b549208380 x1855007616785152/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/4536 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.649155:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549208380 with x1855007616785152 ext(1785856->1789951) 00002000:00010000:2.0:1769074767.649158:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549208380 00000100:00100000:2.0:1769074767.669401:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.669456:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.670548:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616785280 00000100:00100000:0.0:1769074767.670568:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616785280 00002000:00100000:0.0:1769074767.670648:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5476f8e00 x1855007616785280/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.670661:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8e00 with x1855007616785280 ext(0->1048575) 00002000:00010000:0.0:1769074767.670668:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8e00 00000100:00100000:0.0:1769074767.670708:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616785408 00000100:00100000:0.0:1769074767.670712:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616785408 00002000:00100000:0.0:1769074767.670728:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1097727) req@ffff89b5476fb480 x1855007616785408/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.670736:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fb480 with x1855007616785408 ext(1048576->1097727) 00002000:00010000:0.0:1769074767.670751:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476fb480 00000100:00100000:0.0:1769074767.670781:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8441 00000100:00100000:0.0:1769074767.670792:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616785280:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074767.672549:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8442 00000100:00100000:1.0:1769074767.672579:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616785408:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074767.693488:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616785408:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20909us (24013us total) trans 0 rc 49152/49152 00000100:00100000:3.0:1769074767.693507:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8442 00002000:00100000:3.0:1769074767.693523:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1097727) req@ffff89b5476fb480 x1855007616785408/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 49152/49152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.693542:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fb480 with x1855007616785408 ext(1048576->1097727) 00002000:00010000:3.0:1769074767.693550:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476fb480 00000100:00100000:0.0:1769074767.724523:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616785280:12345-192.168.204.60@tcp:3:cat.0 Request processed in 53714us (55132us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074767.724542:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8441 00002000:00100000:0.0:1769074767.724560:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5476f8e00 x1855007616785280/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.724573:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8e00 with x1855007616785280 ext(0->1048575) 00002000:00010000:0.0:1769074767.724582:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8e00 00000100:00100000:1.0:1769074767.735915:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.738078:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616785664 00000100:00100000:0.0:1769074767.738103:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616785664 00000100:00100000:0.0:1769074767.738227:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13304 00000100:00100000:0.0:1769074767.738241:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:62757:x1855007616785664:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074767.738280:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074767.738313:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd53530 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b85d expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.738454:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738474:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738490:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738509:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738517:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738522:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738706:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53545 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738720:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53545 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738727:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53545 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:0.0:1769074767.738741:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074767.738934:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738943:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738952:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738960:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738966:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b437600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074767.738971:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738981:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.738997:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074767.739001:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074767.739008:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739015:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd53537 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739034:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739040:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739047:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739052:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b679924a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074767.739054:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739064:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739071:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47400) 00010000:00010000:0.0:1769074767.739074:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074767.739077:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739082:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924a00/0x7fe2323adfd5353e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074767.739141:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd53530 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b85d expref: 247 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.739157:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53545 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b85d expref: 246 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.739166:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074767.739168:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074767.739171:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924600, rc 0) 00000100:00100000:0.0:1769074767.739286:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+246:62757:x1855007616785664:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1048us (3389us total) trans 8589943915 rc 0/0 00000100:00100000:0.0:1769074767.739295:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13304 00000100:00100000:2.0:1769074767.753053:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.753140:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.753509:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616785792 00000100:00100000:1.0:1769074767.753527:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616785792 00000100:00100000:1.0:1769074767.753621:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 994 00000100:00100000:1.0:1769074767.753633:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+246:2407:x1855007616785792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.753657:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.753694:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924600/0x7fe2323adfd53545 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b85d expref: 246 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.753720:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074767.753727:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.753844:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:2407:x1855007616785792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 212us (702us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.753853:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 994 00000100:00100000:3.0:1769074767.754674:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786048 00000100:00100000:3.0:1769074767.754684:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786048 00000100:00100000:3.0:1769074767.754758:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4652 00000100:00100000:3.0:1769074767.754768:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62757:x1855007616786048:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074767.755174:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+246:62757:x1855007616786048:12345-192.168.204.60@tcp:35:cat.0 Request processed in 406us (2140us total) trans 8589943916 rc 0/0 00000100:00100000:3.0:1769074767.755183:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4652 00000100:00100000:2.0:1769074767.783184:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.783250:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.783352:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786176 00000100:00100000:0.0:1769074767.783367:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786176 00002000:00100000:0.0:1769074767.783459:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1789952->2097151) req@ffff89b5476fa680 x1855007616786176/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.783473:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fa680 with x1855007616786176 ext(1789952->2097151) 00002000:00010000:0.0:1769074767.783481:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476fa680 00000100:00100000:0.0:1769074767.783529:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8443 00000100:00100000:0.0:1769074767.783540:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616786176:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074767.784579:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786304 00000100:00100000:3.0:1769074767.784590:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786304 00000100:00100000:3.0:1769074767.784645:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13939 00000100:00100000:3.0:1769074767.784655:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616786304:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.784672:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.784696:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53553 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b864 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074767.784767:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625113344:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074767.784888:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074767.790142:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616786176:12345-192.168.204.60@tcp:3:cat.0 Request processed in 6582us (6953us total) trans 0 rc 307200/307200 00000100:00100000:0.0:1769074767.790161:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8443 00002000:00100000:0.0:1769074767.790178:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1789952->2097151) req@ffff89b5476fa680 x1855007616786176/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 307200/307200 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074767.790191:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fa680 with x1855007616786176 ext(1789952->2097151) 00002000:00010000:0.0:1769074767.790199:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476fa680 00002000:00010000:3.0:1769074767.792155:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074765 -> 1769074767 00002000:00010000:3.0:1769074767.792169:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074763 -> 1769074767 00002000:00010000:3.0:1769074767.792172:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074765 -> 1769074767 00000100:00100000:3.0:1769074767.792203:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625113344:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074767.792263:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53553 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b864 expref: 23 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.792559:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53553 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b864 expref: 23 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.792574:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:3.0:1769074767.792661:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616786304:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8006us (9408us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074767.792672:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13939 00000100:00100000:2.0:1769074767.802641:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.802695:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.802914:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786432 00000100:00100000:3.0:1769074767.802931:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786432 00000100:00100000:3.0:1769074767.803015:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786560 00000100:00100000:3.0:1769074767.803019:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786560 00000100:00100000:3.0:1769074767.803161:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13940 00000100:00100000:3.0:1769074767.803175:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616786432:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074767.803208:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.803245:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd5355a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b86b expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074767.803353:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625113472:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074767.803495:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074767.804406:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13941 00000100:00100000:1.0:1769074767.804426:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616786560:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074767.804441:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074767.804460:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd53561 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b872 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.804517:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625113600:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074767.804596:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074767.807509:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625113600:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074767.807566:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd53561 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b872 expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074767.807598:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70000/0x7fe2323adfd53561 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b872 expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074767.807613:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70000, rc 0) 00000100:00100000:1.0:1769074767.807699:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616786560:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3280us (5001us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074767.807711:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13941 00000100:00100000:3.0:1769074767.809333:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625113472:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074767.809382:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd5355a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b86b expref: 13 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.809423:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd5355a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b86b expref: 13 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.809444:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:2.0:1769074767.809668:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616786432:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6492us (7039us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074767.809679:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13940 00000100:00100000:2.0:1769074767.823213:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.823475:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786688 00000100:00100000:0.0:1769074767.823492:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786688 00000100:00100000:0.0:1769074767.823579:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13942 00000100:00100000:0.0:1769074767.823592:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b677578850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62757:x1855007616786688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074767.823620:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074767.823649:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b879 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.823690:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.825408:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074767.825421:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074767.825437:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.825672:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 11 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.825694:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 11 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.825716:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625113728:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074767.826045:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:0.0:1769074767.831159:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.831774:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786816 00000100:00100000:1.0:1769074767.831792:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786816 00000100:00100000:1.0:1769074767.831908:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13943 00000100:00100000:1.0:1769074767.831921:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62753:x1855007616786816:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074767.831951:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074767.831979:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b880 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.832020:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.832031:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074767.832033:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074767.832038:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.832480:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.832498:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.832516:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625113856:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074767.832649:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074767.835585:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625113728:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074767.835638:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.835666:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924a00, rc 0) 00000100:00100000:1.0:1769074767.835757:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b677578850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62757:x1855007616786688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12166us (12563us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.835769:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13942 00000100:00100000:3.0:1769074767.842593:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625113856:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074767.842643:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.842671:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70000, rc 0) 00000100:00100000:3.0:1769074767.843414:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62753:x1855007616786816:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11490us (12358us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.843428:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13943 00000100:00100000:2.0:1769074767.843720:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.843861:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.847680:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616786944 00000100:00100000:1.0:1769074767.847706:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616786944 00002000:00100000:1.0:1769074767.848237:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1097728->1789951) req@ffff89b5476fad80 x1855007616786944/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.848252:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fad80 with x1855007616786944 ext(1097728->1789951) 00010000:00010000:1.0:1769074767.848266:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.848505:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 13 pid: 27844 timeout: 1855 lvb_type: 0 00002000:00010000:1.0:1769074767.848515:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5476fad80 00000100:00100000:1.0:1769074767.848565:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616787072 00000100:00100000:1.0:1769074767.848571:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616787072 00002000:00100000:1.0:1769074767.848595:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4431871) req@ffff89b5476f9880 x1855007616787072/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074767.848603:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9880 with x1855007616787072 ext(4194304->4431871) 00010000:00010000:1.0:1769074767.848610:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.848619:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 23 pid: 27844 timeout: 1855 lvb_type: 0 00002000:00010000:1.0:1769074767.848627:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5476f9880 00000100:00100000:1.0:1769074767.848721:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 733 00000100:00100000:1.0:1769074767.848732:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616786944:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074767.850343:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 734 00000100:00100000:2.0:1769074767.850365:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616787072:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074767.879731:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.883196:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616787328 00000100:00100000:2.0:1769074767.883222:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616787328 00000100:00100000:2.0:1769074767.883347:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13305 00000100:00100000:2.0:1769074767.883361:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+246:62759:x1855007616787328:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074767.883415:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074767.883452:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd53576 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b887 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.883554:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883571:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883585:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883601:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883609:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883615:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883755:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd5358b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883765:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd5358b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.883772:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd5358b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074767.883784:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000100:00100000:3.0:1769074767.884504:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616787072:12345-192.168.204.60@tcp:4:cat.0 Request processed in 34141us (40635us total) trans 17179874497 rc 0/0 00000100:00100000:3.0:1769074767.884517:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 734 00002000:00100000:3.0:1769074767.884530:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4431871) req@ffff89b5476f9880 x1855007616787072/t17179874497(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/448 e 0 to 0 dl 1769074778 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.884540:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9880 with x1855007616787072 ext(4194304->4431871) 00010000:00010000:3.0:1769074767.884551:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 25 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:3.0:1769074767.884564:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 25 pid: 27844 timeout: 1855 lvb_type: 0 00002000:00010000:3.0:1769074767.884572:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5476f9880 00000100:00100000:3.0:1769074767.886580:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616786944:12345-192.168.204.60@tcp:4:cat.0 Request processed in 37829us (42842us total) trans 17179874498 rc 0/0 00000100:00100000:3.0:1769074767.886602:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 733 00002000:00100000:3.0:1769074767.886620:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1097728->1789951) req@ffff89b5476fad80 x1855007616786944/t17179874498(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/448 e 0 to 0 dl 1769074778 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.886631:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fad80 with x1855007616786944 ext(1097728->1789951) 00010000:00010000:3.0:1769074767.886644:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 15 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:3.0:1769074767.886663:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 15 pid: 27844 timeout: 1855 lvb_type: 0 00002000:00010000:3.0:1769074767.886671:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5476fad80 00010000:00010000:2.0:1769074767.889712:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889738:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889746:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889755:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889761:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074767.889768:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889776:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889796:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074767.889799:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074767.889840:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889848:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd5357d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889879:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889886:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889891:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889896:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074767.889899:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889904:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889912:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4000) 00010000:00010000:2.0:1769074767.889915:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074767.889918:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.889928:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd53584 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074767.890005:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867800/0x7fe2323adfd53576 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b887 expref: 249 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.890024:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd5358b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b887 expref: 248 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.890033:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074767.890036:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074767.890039:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866800, rc 0) 00000100:00100000:2.0:1769074767.903602:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62759:x1855007616787328:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20221us (23864us total) trans 8589943917 rc 0/0 00000100:00100000:2.0:1769074767.903621:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13305 00000100:00100000:2.0:1769074767.908732:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.910224:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616787456 00000100:00100000:3.0:1769074767.910251:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616787456 00010000:00010000:3.0:1769074767.910793:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 25 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:3.0:1769074767.911365:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6946 00000100:00100000:3.0:1769074767.911382:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616787456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074767.911419:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074767.911454:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 25 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:3.0:1769074767.911473:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 25 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:3.0:1769074767.911514:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd53514 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4431871) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b841 expref: 25 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:3.0:1769074767.911536:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.911538:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.911549:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.911562:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.911616:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074767.911635:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625113984:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074767.912254:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074767.932795:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.932877:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074767.933220:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616787840 00000100:00100000:2.0:1769074767.933234:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616787840 00000100:00100000:2.0:1769074767.933418:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 995 00000100:00100000:2.0:1769074767.933431:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:2407:x1855007616787840:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074767.933453:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074767.933487:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd5358b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b887 expref: 248 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074767.933513:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074767.933518:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074767.933592:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:2407:x1855007616787840:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 162us (711us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.933594:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625113984:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074767.933610:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 995 00010000:00010000:3.0:1769074767.933633:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074767.933696:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616787456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 22318us (24983us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074767.933708:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6946 00000100:00100000:0.0:1769074767.934038:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616787712 00000100:00100000:0.0:1769074767.934045:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616787712 00000100:00100000:0.0:1769074767.934080:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4653 00000100:00100000:0.0:1769074767.934088:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62759:x1855007616787712:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074767.934476:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.934778:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62759:x1855007616787712:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 689us (1998us total) trans 8589943918 rc 0/0 00000100:00100000:0.0:1769074767.934788:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4653 00000100:00100000:3.0:1769074767.936720:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616787968 00000100:00100000:3.0:1769074767.936737:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616787968 00002000:00100000:3.0:1769074767.936799:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5476f8700 x1855007616787968/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.936847:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8700 with x1855007616787968 ext(2097152->3145727) 00002000:00010000:3.0:1769074767.936853:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8700 00000100:00100000:3.0:1769074767.937524:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8444 00000100:00100000:3.0:1769074767.937538:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616787968:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074767.947454:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074767.947751:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788096 00000100:00100000:0.0:1769074767.947766:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788096 00010000:00010000:0.0:1769074767.947856:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 17 pid: 27844 timeout: 1855 lvb_type: 0 00000100:00100000:0.0:1769074767.947903:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6947 00000100:00100000:0.0:1769074767.947914:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616788096:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074767.947932:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074767.947960:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 17 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.947976:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 17 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.948008:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5351b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1097728->1789951) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b848 expref: 17 pid: 27844 timeout: 1855 lvb_type: 0 00010000:00010000:0.0:1769074767.948036:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.948038:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074767.948048:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.948060:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074767.948112:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074767.948129:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625114112:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074767.948240:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074767.964626:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074767.966062:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788224 00000100:00100000:3.0:1769074767.966086:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788224 00000100:00100000:3.0:1769074767.966192:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13944 00000100:00100000:3.0:1769074767.966206:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62759:x1855007616788224:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074767.966236:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074767.966267:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b88e expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.966331:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.966342:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.966351:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.966354:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074767.966360:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [172032->208895] (req 172032->208895) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.966427:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074767.966452:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 16 pid: 27844 timeout: 1855 lvb_type: 1 00000100:00100000:3.0:1769074767.966474:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625114240:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074767.966638:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074767.966649:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 25 pid: 28955 timeout: 1855 lvb_type: 0 00000100:00100000:3.0:1769074767.966659:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625114368:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074767.966686:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074767.972473:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625114112:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074767.972513:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074767.972599:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616788096:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 24684us (25157us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074767.972613:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6947 00000100:00100000:3.0:1769074767.973118:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616787968:12345-192.168.204.60@tcp:3:cat.0 Request processed in 35582us (38640us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074767.973131:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8444 00002000:00100000:3.0:1769074767.973147:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5476f8700 x1855007616787968/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074767.973161:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8700 with x1855007616787968 ext(2097152->3145727) 00002000:00010000:3.0:1769074767.973168:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8700 00000100:00100000:1.0:1769074767.995670:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.995714:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.995773:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074767.995881:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788608 00000100:00100000:1.0:1769074767.995894:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788608 00010000:00010000:1.0:1769074767.995951:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 14 pid: 27844 timeout: 1855 lvb_type: 1 00000100:00100000:1.0:1769074767.995994:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788736 00000100:00100000:1.0:1769074767.995997:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788736 00010000:00010000:1.0:1769074767.996014:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 26 pid: 28955 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074767.996034:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6948 00000100:00100000:1.0:1769074767.996044:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616788608:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.996066:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.996095:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 14 pid: 27844 timeout: 1855 lvb_type: 1 00010000:00010000:1.0:1769074767.996111:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 14 pid: 27844 timeout: 1855 lvb_type: 1 00010000:00010000:1.0:1769074767.996143:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074767.996208:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616788608:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 164us (556us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074767.996216:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6948 00000100:00100000:1.0:1769074767.996240:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6949 00000100:00100000:1.0:1769074767.996247:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616788736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074767.996255:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074767.996262:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 26 pid: 28955 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.996273:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 26 pid: 28955 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074767.996289:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.996292:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074767.996301:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.996312:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074767.996363:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074767.996381:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625114496:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074767.996459:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:2.0:1769074767.996570:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788352 00000100:00100000:2.0:1769074767.996576:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788352 00002000:00100000:2.0:1769074767.996619:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cde7480 x1855007616788352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/0 e 0 to 0 dl 1769074778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074767.996630:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7480 with x1855007616788352 ext(1048576->2097151) 00002000:00010000:2.0:1769074767.996636:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde7480 00000100:00100000:2.0:1769074767.996667:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8445 00000100:00100000:2.0:1769074767.996674:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616788352:12345-192.168.204.60@tcp:3:multiop.0 00010000:00010000:3.0:1769074768.000987:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd534f8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b825 expref: 13 pid: 27844 timeout: 1855 lvb_type: 1 00000100:00100000:3.0:1769074768.001023:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625114240:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074768.001045:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70c00/0x7fe2323adfd534ab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b7f4 expref: 28 pid: 28955 timeout: 1855 lvb_type: 0 00000100:00100000:3.0:1769074768.001057:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625114368:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074768.001106:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 27 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.001127:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:3.0:1769074768.001227:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62759:x1855007616788224:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 35023us (36616us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.001239:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13944 00000100:00100000:1.0:1769074768.002059:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.003464:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788480 00000100:00100000:3.0:1769074768.003480:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788480 00000100:00100000:3.0:1769074768.003553:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13945 00000100:00100000:3.0:1769074768.003565:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62753:x1855007616788480:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074768.003588:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074768.003624:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b895 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.003662:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.003672:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074768.003675:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074768.003679:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.003728:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.003749:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 19 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:3.0:1769074768.003765:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625114624:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074768.005229:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.006039:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625114496:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074768.006067:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.006134:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616788736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9885us (10359us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.006147:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6949 00000100:00100000:1.0:1769074768.009164:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.009332:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788864 00000100:00100000:0.0:1769074768.009347:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788864 00002000:00100000:0.0:1769074768.009429:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (172032->176127) req@ffff89b54cde6a00 x1855007616788864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.009442:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde6a00 with x1855007616788864 ext(172032->176127) 00002000:00010000:0.0:1769074768.009450:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde6a00 00000100:00100000:0.0:1769074768.009495:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8446 00000100:00100000:0.0:1769074768.009508:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616788864:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074768.009762:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616788864:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 255us (613us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074768.009771:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8446 00002000:00100000:0.0:1769074768.009780:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (172032->176127) req@ffff89b54cde6a00 x1855007616788864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.009789:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde6a00 with x1855007616788864 ext(172032->176127) 00002000:00010000:0.0:1769074768.009792:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde6a00 00000100:00100000:1.0:1769074768.012126:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625114624:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.012178:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.012204:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d800, rc 0) 00000100:00100000:1.0:1769074768.012279:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62753:x1855007616788480:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8715us (10225us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.012289:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13945 00000100:00100000:2.0:1769074768.014085:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616788352:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 17397us (18356us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074768.014103:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8445 00002000:00100000:2.0:1769074768.014119:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cde7480 x1855007616788352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:303/0 lens 488/440 e 0 to 0 dl 1769074778 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.014133:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7480 with x1855007616788352 ext(1048576->2097151) 00002000:00010000:2.0:1769074768.014140:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde7480 00000100:00100000:3.0:1769074768.016555:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.018519:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616788992 00000100:00100000:2.0:1769074768.018539:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616788992 00002000:00100000:2.0:1769074768.018643:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3784704->4194303) req@ffff89b54d100e00 x1855007616788992/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.018656:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100e00 with x1855007616788992 ext(3784704->4194303) 00010000:00010000:2.0:1769074768.018669:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 20 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:2.0:1769074768.018687:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 20 pid: 8398 timeout: 1855 lvb_type: 0 00002000:00010000:2.0:1769074768.018696:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d100e00 00000100:00100000:2.0:1769074768.018728:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 735 00000100:00100000:2.0:1769074768.018754:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616788992:12345-192.168.204.60@tcp:4:multiop.0 00000100:00080000:3.0:1769074768.018837:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074768.023799:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.024007:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789120 00000100:00100000:0.0:1769074768.024022:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789120 00002000:00100000:0.0:1769074768.024091:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (204800->208895) req@ffff89b54cde4380 x1855007616789120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.024104:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4380 with x1855007616789120 ext(204800->208895) 00002000:00010000:0.0:1769074768.024112:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde4380 00000100:00100000:0.0:1769074768.024153:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8447 00000100:00100000:0.0:1769074768.024165:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616789120:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074768.024534:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616789120:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 369us (748us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074768.024544:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8447 00002000:00100000:0.0:1769074768.024553:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (204800->208895) req@ffff89b54cde4380 x1855007616789120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.024561:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4380 with x1855007616789120 ext(204800->208895) 00002000:00010000:0.0:1769074768.024565:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde4380 00000100:00100000:3.0:1769074768.041508:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.041981:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789248 00000100:00100000:0.0:1769074768.042015:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789248 00000100:00100000:0.0:1769074768.042246:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13946 00000100:00100000:0.0:1769074768.042324:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62757:x1855007616789248:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074768.042401:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074768.042429:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b89c expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.042469:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.042472:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.042484:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.042502:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.042520:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437600, rc 0) 00000100:00100000:0.0:1769074768.044153:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62757:x1855007616789248:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1882us (2649us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.044167:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13946 00000100:00100000:2.0:1769074768.049224:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616788992:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 30452us (32668us total) trans 21474841183 rc 0/0 00000100:00100000:2.0:1769074768.049249:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 735 00002000:00100000:2.0:1769074768.049274:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3784704->4194303) req@ffff89b54d100e00 x1855007616788992/t21474841183(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/448 e 0 to 0 dl 1769074779 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.049287:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100e00 with x1855007616788992 ext(3784704->4194303) 00010000:00010000:2.0:1769074768.049302:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 22 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:2.0:1769074768.049329:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 22 pid: 8398 timeout: 1855 lvb_type: 0 00002000:00010000:2.0:1769074768.049339:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d100e00 00000100:00100000:3.0:1769074768.066544:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.066658:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.066700:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.067222:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789504 00000100:00100000:0.0:1769074768.067238:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789504 00000100:00100000:0.0:1769074768.067340:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13947 00000100:00100000:0.0:1769074768.067352:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62761:x1855007616789504:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074768.067378:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074768.067427:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8a3 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.067481:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074768.067484:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074768.067491:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3944448->4149247] (req 3944448->4149247) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.067508:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3944448->4149247] (req 3944448->4149247) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.067527:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437e00, rc 0) 00000100:00100000:0.0:1769074768.067610:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62761:x1855007616789504:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 260us (951us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.067619:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13947 00000100:00100000:1.0:1769074768.069163:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789632 00000100:00100000:1.0:1769074768.069201:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789632 00010000:00010000:1.0:1769074768.069565:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 22 pid: 8398 timeout: 1855 lvb_type: 0 00000100:00100000:1.0:1769074768.069616:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6950 00000100:00100000:1.0:1769074768.069628:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616789632:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074768.069649:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.069695:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 22 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074768.069721:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 22 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074768.070619:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd534ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3784704->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b82c expref: 22 pid: 8398 timeout: 1855 lvb_type: 0 00010000:00010000:1.0:1769074768.070643:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.070646:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.070656:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.070667:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.070678:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.070679:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.070683:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.070690:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.071836:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.071871:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625114752:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074768.072055:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.072066:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625114880:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.072099:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074768.072663:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789376 00000100:00100000:2.0:1769074768.072682:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789376 00002000:00100000:2.0:1769074768.072774:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54d103480 x1855007616789376/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.072793:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103480 with x1855007616789376 ext(2097152->3145727) 00002000:00010000:2.0:1769074768.072800:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d103480 00000100:00100000:2.0:1769074768.072901:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8448 00000100:00100000:2.0:1769074768.072913:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616789376:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074768.089636:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625114752:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.089659:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625114880:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074768.089691:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074768.089768:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616789632:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 20142us (23064us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.089783:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6950 00000100:00100000:1.0:1769074768.093338:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.093704:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789760 00000100:00100000:0.0:1769074768.093721:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789760 00002000:00100000:0.0:1769074768.093853:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (3944448->3948543) req@ffff89b5475d8700 x1855007616789760/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.093868:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8700 with x1855007616789760 ext(3944448->3948543) 00002000:00010000:0.0:1769074768.093876:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475d8700 00000100:00100000:0.0:1769074768.093909:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8449 00000100:00100000:0.0:1769074768.093920:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616789760:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074768.094215:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616789760:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 294us (892us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074768.094226:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8449 00002000:00100000:0.0:1769074768.094235:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (3944448->3948543) req@ffff89b5475d8700 x1855007616789760/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.094244:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8700 with x1855007616789760 ext(3944448->3948543) 00002000:00010000:0.0:1769074768.094248:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475d8700 00000100:00100000:2.0:1769074768.099637:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616789376:12345-192.168.204.60@tcp:3:cat.0 Request processed in 26707us (33091us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074768.099660:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8448 00002000:00100000:2.0:1769074768.099679:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54d103480 x1855007616789376/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.099692:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103480 with x1855007616789376 ext(2097152->3145727) 00002000:00010000:2.0:1769074768.099699:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d103480 00000100:00100000:2.0:1769074768.103146:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.103494:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616789888 00000100:00100000:3.0:1769074768.103509:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616789888 00002000:00100000:3.0:1769074768.103579:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5476f9500 x1855007616789888/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.103593:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9500 with x1855007616789888 ext(3145728->4194303) 00002000:00010000:3.0:1769074768.103601:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f9500 00000100:00100000:3.0:1769074768.103694:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8450 00000100:00100000:3.0:1769074768.103706:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616789888:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074768.110557:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.111583:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790016 00000100:00100000:0.0:1769074768.111601:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790016 00002000:00100000:0.0:1769074768.111671:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54d103800 x1855007616790016/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.111684:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103800 with x1855007616790016 ext(2097152->3145727) 00002000:00010000:0.0:1769074768.111691:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d103800 00000100:00100000:0.0:1769074768.111742:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8451 00000100:00100000:0.0:1769074768.111753:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616790016:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074768.114933:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.117203:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790144 00000100:00100000:3.0:1769074768.117231:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790144 00002000:00100000:3.0:1769074768.117314:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (4145152->4149247) req@ffff89b5476fb100 x1855007616790144/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.117329:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fb100 with x1855007616790144 ext(4145152->4149247) 00002000:00010000:3.0:1769074768.117337:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5476fb100 00000100:00100000:0.0:1769074768.117915:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8452 00000100:00100000:0.0:1769074768.117933:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616790144:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074768.119245:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616790144:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 1309us (4324us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074768.119257:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8452 00002000:00100000:0.0:1769074768.119269:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (4145152->4149247) req@ffff89b5476fb100 x1855007616790144/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.119280:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fb100 with x1855007616790144 ext(4145152->4149247) 00002000:00010000:0.0:1769074768.119287:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5476fb100 00000100:00100000:1.0:1769074768.126873:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616789888:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 23151us (23725us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074768.126893:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8450 00002000:00100000:1.0:1769074768.126912:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5476f9500 x1855007616789888/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.126924:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9500 with x1855007616789888 ext(3145728->4194303) 00002000:00010000:1.0:1769074768.126932:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f9500 00000100:00100000:3.0:1769074768.131307:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.131582:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790272 00000100:00100000:1.0:1769074768.131596:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790272 00000100:00100000:1.0:1769074768.131837:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13948 00000100:00100000:1.0:1769074768.131852:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62757:x1855007616790272:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.131878:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.131904:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8aa expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.131941:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.131944:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.131956:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.131977:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.131994:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:1.0:1769074768.132990:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62757:x1855007616790272:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1133us (1690us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.133002:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13948 00000100:00100000:0.0:1769074768.143167:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616790016:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 31393us (32602us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074768.143188:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8451 00002000:00100000:0.0:1769074768.143216:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54d103800 x1855007616790016/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.143229:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103800 with x1855007616790016 ext(2097152->3145727) 00002000:00010000:0.0:1769074768.143238:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d103800 00000100:00100000:3.0:1769074768.157198:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.159280:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.160659:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790400 00000100:00100000:0.0:1769074768.160690:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790400 00002000:00100000:0.0:1769074768.160801:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54d101500 x1855007616790400/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.160849:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101500 with x1855007616790400 ext(2097152->3145727) 00002000:00010000:0.0:1769074768.160858:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d101500 00000100:00100000:1.0:1769074768.161144:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790528 00000100:00100000:1.0:1769074768.161152:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790528 00002000:00100000:1.0:1769074768.161189:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54d100a80 x1855007616790528/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.161210:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100a80 with x1855007616790528 ext(3145728->4194303) 00002000:00010000:1.0:1769074768.161215:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d100a80 00000100:00100000:1.0:1769074768.161278:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8453 00000100:00100000:1.0:1769074768.161291:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616790400:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074768.161586:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8454 00000100:00100000:3.0:1769074768.161599:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616790528:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074768.167629:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.167783:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790656 00000100:00100000:3.0:1769074768.167799:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790656 00000100:00100000:3.0:1769074768.167905:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13949 00000100:00100000:3.0:1769074768.167917:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62753:x1855007616790656:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074768.167942:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074768.167970:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8b1 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.168712:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 16 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.170953:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074768.170958:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074768.170973:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 16 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.173240:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 16 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.173269:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.173291:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625115008:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074768.173446:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074768.181750:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.181857:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790784 00000100:00100000:0.0:1769074768.181872:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790784 00002000:00100000:0.0:1769074768.181946:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (3944448->4149247) req@ffff89b67fb95c00 x1855007616790784/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.181959:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95c00 with x1855007616790784 ext(3944448->4149247) 00010000:00010000:0.0:1769074768.181970:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.181974:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625115008:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074768.181988:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:0.0:1769074768.181996:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fb95c00 00010000:00010000:1.0:1769074768.182024:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074768.182025:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 736 00000100:00100000:0.0:1769074768.182036:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616790784:12345-192.168.204.60@tcp:4:multiop.0 00010000:00010000:1.0:1769074768.182051:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:1.0:1769074768.182118:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62753:x1855007616790656:12345-192.168.204.60@tcp:101:cat.0 Request processed in 14204us (14507us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.182127:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13949 00000100:00100000:3.0:1769074768.186178:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616790528:12345-192.168.204.60@tcp:3:cat.0 Request processed in 24562us (26896us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074768.186197:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8454 00002000:00100000:3.0:1769074768.186212:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54d100a80 x1855007616790528/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.186221:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100a80 with x1855007616790528 ext(3145728->4194303) 00002000:00010000:3.0:1769074768.186228:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d100a80 00000100:00100000:3.0:1769074768.193606:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616790400:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32302us (36423us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074768.193622:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8453 00002000:00100000:3.0:1769074768.193640:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54d101500 x1855007616790400/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.193652:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101500 with x1855007616790400 ext(2097152->3145727) 00002000:00010000:3.0:1769074768.193659:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d101500 00000100:00100000:1.0:1769074768.196246:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616790784:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 14195us (14497us total) trans 21474841184 rc 0/0 00000100:00100000:1.0:1769074768.196263:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 736 00002000:00100000:1.0:1769074768.196279:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (3944448->4149247) req@ffff89b67fb95c00 x1855007616790784/t21474841184(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/448 e 0 to 0 dl 1769074779 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.196287:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95c00 with x1855007616790784 ext(3944448->4149247) 00010000:00010000:1.0:1769074768.196300:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.196319:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:1.0:1769074768.196326:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fb95c00 00000100:00100000:0.0:1769074768.203885:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.204150:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616790912 00000100:00100000:1.0:1769074768.204166:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616790912 00010000:00010000:1.0:1769074768.204368:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.204442:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6951 00000100:00100000:1.0:1769074768.204455:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616790912:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074768.204479:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.204512:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.204528:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.204562:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd535ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3944448->18446744073709551615] (req 3944448->4149247) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8a3 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.204583:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.204585:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.204595:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.204608:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.204654:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.204670:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625115136:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.204859:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074768.207303:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.207555:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791040 00000100:00100000:1.0:1769074768.207570:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791040 00000100:00100000:1.0:1769074768.207721:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13950 00000100:00100000:1.0:1769074768.207743:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62757:x1855007616791040:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.207774:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.207846:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8b8 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.207884:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.207887:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.207900:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.207922:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.207937:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641520e00, rc 0) 00000100:00100000:1.0:1769074768.208029:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62757:x1855007616791040:12345-192.168.204.60@tcp:101:cat.0 Request processed in 296us (743us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.208037:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13950 00000100:00100000:2.0:1769074768.212035:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.212168:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791168 00000100:00100000:0.0:1769074768.212189:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791168 00002000:00100000:0.0:1769074768.212280:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67fb95180 x1855007616791168/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.212294:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95180 with x1855007616791168 ext(3145728->4194303) 00002000:00010000:0.0:1769074768.212304:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fb95180 00000100:00100000:0.0:1769074768.212351:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8455 00000100:00100000:0.0:1769074768.212371:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616791168:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074768.215312:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625115136:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074768.215354:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074768.215458:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616790912:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11004us (11592us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.215471:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6951 00000100:00100000:2.0:1769074768.236332:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.236686:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791296 00000100:00100000:3.0:1769074768.236712:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791296 00002000:00100000:3.0:1769074768.236861:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67fb95f80 x1855007616791296/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.236877:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95f80 with x1855007616791296 ext(3145728->4194303) 00002000:00010000:3.0:1769074768.236885:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fb95f80 00000100:00100000:3.0:1769074768.236921:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8456 00000100:00100000:3.0:1769074768.236932:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616791296:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074768.237534:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616791168:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 25168us (25522us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074768.237546:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8455 00002000:00100000:0.0:1769074768.237555:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67fb95180 x1855007616791168/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.237565:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95180 with x1855007616791168 ext(3145728->4194303) 00002000:00010000:0.0:1769074768.237571:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fb95180 00000100:00100000:3.0:1769074768.250867:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.257618:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.260585:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616791296:12345-192.168.204.60@tcp:3:cat.0 Request processed in 23638us (24253us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074768.260604:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8456 00002000:00100000:3.0:1769074768.260622:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67fb95f80 x1855007616791296/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.260635:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95f80 with x1855007616791296 ext(3145728->4194303) 00002000:00010000:3.0:1769074768.260642:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fb95f80 00000100:00100000:1.0:1769074768.261632:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791424 00000100:00100000:1.0:1769074768.261653:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791424 00000100:00100000:1.0:1769074768.261912:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791552 00000100:00100000:1.0:1769074768.261917:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791552 00000100:00100000:1.0:1769074768.261954:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13951 00000100:00100000:1.0:1769074768.261965:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616791424:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.261990:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1769074768.262187:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13952 00000100:00100000:3.0:1769074768.262197:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616791552:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074768.262206:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074768.262221:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8c6 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.262251:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.262254:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.262266:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 15 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074768.262287:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 15 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074768.262308:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c600, rc 0) 00000100:00100000:3.0:1769074768.262402:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616791552:12345-192.168.204.60@tcp:101:cat.0 Request processed in 206us (4800us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.262410:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13952 00010000:00010000:1.0:1769074768.263298:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd535d1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8bf expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.263526:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625115264:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074768.263630:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00002000:00010000:1.0:1769074768.277315:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074766 -> 1769074767 00002000:00010000:1.0:1769074768.277329:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074766 -> 1769074767 00002000:00010000:1.0:1769074768.277332:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074766 -> 1769074767 00000100:00100000:1.0:1769074768.277363:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625115264:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074768.277435:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd535d1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8bf expref: 15 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.277465:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641521e00/0x7fe2323adfd535d1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8bf expref: 15 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.277483:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521e00, rc 0) 00000100:00100000:1.0:1769074768.277569:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616791424:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15604us (26717us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074768.277580:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13951 00010000:00010000:0.0:1769074768.285036:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074768.300724:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.300929:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.302677:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791680 00000100:00100000:1.0:1769074768.302708:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791680 00000100:00100000:2.0:1769074768.303228:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616791808 00000100:00100000:2.0:1769074768.303236:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616791808 00002000:00100000:2.0:1769074768.303311:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b6491ef800 x1855007616791808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.303326:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ef800 with x1855007616791808 ext(4194304->4808703) 00002000:00010000:2.0:1769074768.303334:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ef800 00000100:00100000:0.0:1769074768.303576:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8457 00000100:00100000:0.0:1769074768.303593:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616791808:12345-192.168.204.60@tcp:3:cat.0 00002000:00100000:3.0:1769074768.304231:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54d100000 x1855007616791680/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.304243:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100000 with x1855007616791680 ext(4194304->4808703) 00002000:00010000:3.0:1769074768.304249:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d100000 00000100:00100000:3.0:1769074768.304284:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8458 00000100:00100000:3.0:1769074768.304292:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616791680:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074768.334445:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616791808:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30839us (33498us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074768.334466:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8457 00002000:00100000:0.0:1769074768.334486:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b6491ef800 x1855007616791808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.334505:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6491ef800 with x1855007616791808 ext(4194304->4808703) 00002000:00010000:0.0:1769074768.334514:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6491ef800 00000100:00100000:3.0:1769074768.337558:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.337708:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792064 00000100:00100000:2.0:1769074768.337722:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792064 00000100:00100000:0.0:1769074768.338312:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13306 00000100:00100000:0.0:1769074768.338331:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62767:x1855007616792064:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074768.338372:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074768.338417:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd535d8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8cd expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.338527:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338544:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338557:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338575:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338583:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338589:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338769:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd535ed lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338782:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd535ed lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.338790:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd535ed lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:0.0:1769074768.338801:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000100:00080000:3.0:1769074768.338955:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1769074768.339883:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339897:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339904:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339911:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339918:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074768.339924:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339933:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339969:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074768.339973:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074768.339979:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.339985:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd535df lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340019:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340030:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340037:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340042:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074768.340046:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340061:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340068:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47700) 00010000:00010000:0.0:1769074768.340071:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074768.340074:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340080:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd535e6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074768.340148:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd535d8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8cd expref: 251 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.340166:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd535ed lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8cd expref: 250 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.340174:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074768.340176:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074768.340179:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877800, rc 0) 00000100:00100000:0.0:1769074768.340290:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+250:62767:x1855007616792064:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1964us (2748us total) trans 8589943919 rc 0/0 00000100:00100000:0.0:1769074768.340302:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13306 00000100:00100000:0.0:1769074768.341640:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616791680:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37339us (40923us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074768.341654:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8458 00002000:00100000:0.0:1769074768.341669:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54d100000 x1855007616791680/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.341682:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d100000 with x1855007616791680 ext(4194304->4808703) 00002000:00010000:0.0:1769074768.341688:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d100000 00000100:00100000:3.0:1769074768.343668:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.343755:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792192 00000100:00100000:1.0:1769074768.343773:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792192 00000100:00100000:1.0:1769074768.344714:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13953 00000100:00100000:1.0:1769074768.344739:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616792192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.344765:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.344791:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd535fb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8d4 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.344906:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625115392:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074768.345063:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.346531:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.346579:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.346621:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792448 00000100:00100000:1.0:1769074768.346637:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792448 00000100:00100000:1.0:1769074768.346739:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4654 00000100:00100000:1.0:1769074768.346752:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+250:62767:x1855007616792448:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074768.348617:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62767:x1855007616792448:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 1854us (2082us total) trans 8589943920 rc 0/0 00000100:00100000:1.0:1769074768.348632:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4654 00000100:00100000:1.0:1769074768.348706:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792576 00000100:00100000:1.0:1769074768.348715:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792576 00000100:00100000:1.0:1769074768.348782:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 996 00000100:00100000:1.0:1769074768.348792:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:2407:x1855007616792576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074768.348807:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.349508:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd535ed lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8cd expref: 251 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.349545:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074768.349552:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074768.349618:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+250:2407:x1855007616792576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 824us (3034us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.349629:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 996 00000100:00100000:1.0:1769074768.351546:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625115392:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074768.351602:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd535fb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8d4 expref: 15 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.351630:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd535fb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8d4 expref: 15 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.351645:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f71400, rc 0) 00000100:00100000:1.0:1769074768.351757:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616792192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7030us (8099us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074768.351767:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13953 00000001:00080000:0.0:1769074768.359100:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943919 is committed 00000001:00080000:0.0:1769074768.359143:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943906 is committed 00000001:00080000:0.0:1769074768.359154:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943901 is committed 00000100:00100000:0.0:1769074768.373415:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.373501:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792704 00000100:00100000:1.0:1769074768.373519:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792704 00000100:00100000:1.0:1769074768.373676:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13954 00000100:00100000:1.0:1769074768.373688:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616792704:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.373724:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.373761:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53602 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8db expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.373905:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625115520:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074768.374088:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.375649:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.380753:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792832 00000100:00100000:0.0:1769074768.380775:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792832 00000100:00100000:1.0:1769074768.380959:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13955 00000100:00100000:1.0:1769074768.380973:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62766:x1855007616792832:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074768.380992:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.381014:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8e2 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.381062:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.381072:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.381079:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.381081:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.381084:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [2039808->2097151] (req 2039808->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.381121:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.381136:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 22 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.381149:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625115648:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.381267:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.381277:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.381285:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625115776:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.381310:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.383748:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625115520:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074768.383790:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53602 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8db expref: 15 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.383859:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53602 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8db expref: 15 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.383873:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f71400, rc 0) 00000100:00100000:1.0:1769074768.383947:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616792704:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10261us (10546us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074768.383956:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13954 00000100:00100000:0.0:1769074768.394025:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.394179:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625115648:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.394196:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625115776:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.394245:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [2039808->2097151] (req 2039808->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.394271:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70c00, rc 0) 00000100:00100000:1.0:1769074768.394343:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62766:x1855007616792832:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 13372us (18702us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.394355:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13955 00000100:00100000:3.0:1769074768.394516:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616792960 00000100:00100000:3.0:1769074768.394525:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616792960 00010000:00010000:3.0:1769074768.394569:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.394607:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6952 00000100:00100000:3.0:1769074768.394619:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616792960:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.394636:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.394662:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.394673:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.394698:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd533f5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b776 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.394711:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.394735:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616792960:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 119us (728us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.394741:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6952 00000100:00100000:0.0:1769074768.395476:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.395566:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793088 00000100:00100000:1.0:1769074768.395577:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793088 00010000:00010000:1.0:1769074768.395616:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.395667:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6953 00000100:00100000:1.0:1769074768.395678:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616793088:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074768.395690:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.395707:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.395721:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.395742:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4000/0x7fe2323adfd533fc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b77d expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.395756:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.395758:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.395766:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 16 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.395776:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 16 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.395844:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 16 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.395859:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625115904:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.395941:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.402428:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625115904:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074768.402470:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074768.402573:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616793088:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6893us (7094us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.402589:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6953 00000100:00100000:0.0:1769074768.407719:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.407927:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793216 00000100:00100000:3.0:1769074768.407938:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793216 00002000:00100000:3.0:1769074768.407989:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (2039808->2043903) req@ffff89b54920aa00 x1855007616793216/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.408001:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920aa00 with x1855007616793216 ext(2039808->2043903) 00002000:00010000:3.0:1769074768.408007:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54920aa00 00000100:00100000:3.0:1769074768.408044:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8459 00000100:00100000:3.0:1769074768.408055:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616793216:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074768.408309:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616793216:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 253us (598us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074768.408323:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8459 00002000:00100000:3.0:1769074768.408331:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (2039808->2043903) req@ffff89b54920aa00 x1855007616793216/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.408340:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920aa00 with x1855007616793216 ext(2039808->2043903) 00002000:00010000:3.0:1769074768.408343:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54920aa00 00000100:00100000:0.0:1769074768.420969:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.421246:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793344 00000100:00100000:1.0:1769074768.421257:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793344 00000100:00100000:1.0:1769074768.421339:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13956 00000100:00100000:1.0:1769074768.421350:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62766:x1855007616793344:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074768.421390:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.421424:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8e9 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.421475:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.421486:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 15 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.421492:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.421495:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.421499:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->1429503] (req 1048576->1429503) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 15 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.421541:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 15 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074768.421556:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 15 pid: 10284 timeout: 1856 lvb_type: 1 00000100:00100000:1.0:1769074768.421572:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116032:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.421682:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.421693:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 24 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.421705:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116160:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.421743:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.426520:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.426606:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793472 00000100:00100000:3.0:1769074768.426618:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793472 00010000:00010000:3.0:1769074768.426659:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 16 pid: 10284 timeout: 1856 lvb_type: 1 00000100:00100000:3.0:1769074768.426707:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6954 00000100:00100000:3.0:1769074768.426716:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616793472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.426745:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.426768:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 16 pid: 10284 timeout: 1856 lvb_type: 1 00010000:00010000:3.0:1769074768.426781:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 16 pid: 10284 timeout: 1856 lvb_type: 1 00010000:00010000:3.0:1769074768.426856:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.426903:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616793472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 187us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.426911:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6954 00010000:00010000:2.0:1769074768.427597:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd535ca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8c6 expref: 15 pid: 10284 timeout: 1856 lvb_type: 1 00000100:00100000:2.0:1769074768.427613:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116032:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074768.427626:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116160:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074768.427653:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->1429503] (req 1048576->1429503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 14 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.427665:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f71400, rc 0) 00000100:00100000:2.0:1769074768.427717:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62766:x1855007616793344:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6367us (6759us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.427747:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13956 00000100:00100000:1.0:1769074768.432119:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.432323:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793600 00000100:00100000:3.0:1769074768.432334:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793600 00010000:00010000:3.0:1769074768.432377:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 25 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.432431:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6955 00000100:00100000:3.0:1769074768.432454:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616793600:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.432469:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.432492:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 25 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.432505:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 25 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.432528:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b679924a00/0x7fe2323adfd53568 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b879 expref: 25 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.432544:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.432546:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.432555:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.432565:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.432605:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074768.432619:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625116288:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074768.432721:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8b40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.442142:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625116288:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074768.442177:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.442261:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616793600:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9819us (10154us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.442273:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6955 00000100:00100000:0.0:1769074768.445375:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.445961:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793728 00000100:00100000:1.0:1769074768.445978:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793728 00002000:00100000:1.0:1769074768.446055:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1425408->1429503) req@ffff89b549201c00 x1855007616793728/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.446069:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549201c00 with x1855007616793728 ext(1425408->1429503) 00002000:00010000:1.0:1769074768.446078:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549201c00 00000100:00100000:1.0:1769074768.446122:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8460 00000100:00100000:1.0:1769074768.446137:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616793728:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074768.446400:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616793728:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 264us (1035us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074768.446410:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8460 00002000:00100000:1.0:1769074768.446420:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1425408->1429503) req@ffff89b549201c00 x1855007616793728/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.446431:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549201c00 with x1855007616793728 ext(1425408->1429503) 00002000:00010000:1.0:1769074768.446435:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549201c00 00000100:00100000:2.0:1769074768.534973:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.535081:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793856 00000100:00100000:1.0:1769074768.535091:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793856 00000100:00100000:1.0:1769074768.535151:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13957 00000100:00100000:1.0:1769074768.535160:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62767:x1855007616793856:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074768.535176:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.535202:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8f0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.535231:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 16 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.535240:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.535247:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.535249:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.535253:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3481600->4194303] (req 3481600->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.535284:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.535297:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.535317:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625116416:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.535398:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 16 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.535407:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 16 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.535416:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625116544:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.535438:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.538246:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625116416:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.538262:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625116544:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.538297:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3481600->4194303] (req 3481600->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.538319:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70400, rc 0) 00000100:00100000:1.0:1769074768.538401:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62767:x1855007616793856:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 3240us (3435us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.538411:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13957 00000100:00100000:0.0:1769074768.544624:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.544669:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.544758:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616793984 00000100:00100000:3.0:1769074768.544770:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616793984 00010000:00010000:3.0:1769074768.544859:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.544898:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616794112 00000100:00100000:3.0:1769074768.544903:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616794112 00010000:00010000:3.0:1769074768.544917:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.544939:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6956 00000100:00100000:3.0:1769074768.544947:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616793984:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.544963:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.544986:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.544999:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.545021:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641520e00/0x7fe2323adfd535c3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b8 expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.545047:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.545105:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616793984:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 158us (493us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.545112:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6956 00000100:00100000:3.0:1769074768.545130:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6957 00000100:00100000:3.0:1769074768.545136:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616794112:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.545144:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.545153:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.545162:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.545175:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd535bc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8b1 expref: 17 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.545184:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.545185:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.545193:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.545213:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.545249:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074768.545264:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625116672:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074768.545312:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8b40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.548620:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625116672:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074768.548649:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.548776:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616794112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3637us (4104us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.548788:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6957 00000100:00100000:0.0:1769074768.551957:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.552308:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616794240 00000100:00100000:1.0:1769074768.552322:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616794240 00002000:00100000:1.0:1769074768.552388:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3481600->3485695) req@ffff89b549200000 x1855007616794240/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.552400:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549200000 with x1855007616794240 ext(3481600->3485695) 00002000:00010000:1.0:1769074768.552407:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549200000 00000100:00100000:1.0:1769074768.552441:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8461 00000100:00100000:1.0:1769074768.552451:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616794240:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074768.552632:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616794240:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 181us (685us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074768.552640:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8461 00002000:00100000:1.0:1769074768.552647:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3481600->3485695) req@ffff89b549200000 x1855007616794240/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.552654:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549200000 with x1855007616794240 ext(3481600->3485695) 00002000:00010000:1.0:1769074768.552657:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549200000 00000100:00100000:0.0:1769074768.583785:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.584330:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616794368 00000100:00100000:1.0:1769074768.584343:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616794368 00000100:00100000:1.0:1769074768.584415:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13958 00000100:00100000:1.0:1769074768.584424:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62767:x1855007616794368:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074768.584443:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.584464:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8f7 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.584495:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.584504:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.584510:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.584512:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.584517:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4239359] (req 4194304->4239359) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.584553:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.584565:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.584580:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116800:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.584685:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.584695:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 13 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.584704:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116928:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.585090:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.617674:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116800:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074768.617692:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625116928:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.617726:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4239359] (req 4194304->4239359) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.617756:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:1.0:1769074768.617866:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62767:x1855007616794368:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 33442us (34090us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.617877:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13958 00000100:00100000:0.0:1769074768.620915:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.621028:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616794496 00000100:00100000:3.0:1769074768.621040:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616794496 00010000:00010000:3.0:1769074768.621081:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.621124:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6958 00000100:00100000:3.0:1769074768.621135:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616794496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.621152:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.621176:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.621189:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.621226:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd5356f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b880 expref: 14 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.621251:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.621304:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616794496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 171us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.621312:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6958 00000100:00100000:0.0:1769074768.630741:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.631440:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616794752 00000100:00100000:1.0:1769074768.631455:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616794752 00000100:00100000:1.0:1769074768.631542:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13307 00000100:00100000:1.0:1769074768.631554:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:62775:x1855007616794752:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.631586:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.631623:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53625 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b8fe expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.631710:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631739:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631754:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631769:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631776:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631781:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631940:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5363a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631951:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5363a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.631959:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5363a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074768.631968:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074768.632045:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632052:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632058:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632064:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632070:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074768.632074:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632081:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632092:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074768.632095:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074768.632100:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632106:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5362c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632120:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632128:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632133:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632137:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074768.632140:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632144:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632150:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615200) 00010000:00010000:1.0:1769074768.632152:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074768.632155:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632160:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53633 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074768.632205:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53625 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8fe expref: 228 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.632218:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5363a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8fe expref: 227 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.632225:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074768.632227:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074768.632230:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:1.0:1769074768.632318:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62775:x1855007616794752:12345-192.168.204.60@tcp:101:cat.0 Request processed in 765us (1589us total) trans 8589943921 rc 0/0 00000100:00100000:1.0:1769074768.632326:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13307 00000100:00100000:1.0:1769074768.645427:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.645488:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.645525:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.645730:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616794880 00000100:00100000:0.0:1769074768.645744:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616794880 00010000:00010000:0.0:1769074768.645803:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074768.645881:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795264 00000100:00100000:0.0:1769074768.645885:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795264 00000100:00100000:0.0:1769074768.645915:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6959 00000100:00100000:0.0:1769074768.645925:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616794880:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074768.645942:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074768.645970:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.645986:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.646015:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437600/0x7fe2323adfd535a7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b89c expref: 25 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.646035:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.646037:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.646046:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.646057:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.646112:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 24 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074768.646133:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625117056:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074768.646229:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074768.646252:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 997 00000100:00100000:0.0:1769074768.646260:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2409:x1855007616795264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074768.646272:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074768.646289:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5363a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8fe expref: 227 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.646305:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074768.646310:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074768.646335:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2409:x1855007616795264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 77us (810us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.646344:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 997 00000100:00100000:3.0:1769074768.647304:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795136 00000100:00100000:3.0:1769074768.647315:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795136 00000100:00100000:3.0:1769074768.647367:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4655 00000100:00100000:3.0:1769074768.647385:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c4050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:62775:x1855007616795136:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074768.647663:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c4050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62775:x1855007616795136:12345-192.168.204.60@tcp:35:cat.0 Request processed in 287us (2256us total) trans 8589943922 rc 0/0 00000100:00100000:3.0:1769074768.647671:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4655 00000100:00100000:3.0:1769074768.662021:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.662070:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.662105:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.662128:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.662246:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795392 00000100:00100000:1.0:1769074768.662260:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795392 00002000:00100000:1.0:1769074768.662323:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4235264->4239359) req@ffff89b547598e00 x1855007616795392/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.662335:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547598e00 with x1855007616795392 ext(4235264->4239359) 00002000:00010000:1.0:1769074768.662342:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547598e00 00000100:00100000:1.0:1769074768.662388:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8462 00000100:00100000:1.0:1769074768.662399:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616795392:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074768.662584:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616795392:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 186us (572us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074768.662591:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8462 00002000:00100000:1.0:1769074768.662597:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4235264->4239359) req@ffff89b547598e00 x1855007616795392/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.662618:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547598e00 with x1855007616795392 ext(4235264->4239359) 00002000:00010000:1.0:1769074768.662622:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547598e00 00000100:00100000:1.0:1769074768.662646:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795520 00000100:00100000:1.0:1769074768.662650:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795520 00000100:00100000:1.0:1769074768.662674:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795776 00000100:00100000:1.0:1769074768.662678:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795776 00000100:00100000:1.0:1769074768.662701:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795648 00000100:00100000:1.0:1769074768.662703:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795648 00000100:00100000:1.0:1769074768.662723:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13959 00000100:00100000:1.0:1769074768.662728:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616795520:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.662737:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.662759:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53648 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b905 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.662857:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625117184:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074768.662922:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.662941:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13960 00000100:00100000:1.0:1769074768.662948:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616795776:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074768.662956:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074768.662963:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5364f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b913 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.662991:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625117312:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074768.663016:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648cb40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074768.664050:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13961 00000100:00100000:2.0:1769074768.664073:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616795648:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074768.664091:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.664112:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53656 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b90c expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074768.664167:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625117440:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074768.664273:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074768.669106:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625117056:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074768.669143:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074768.669252:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616794880:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 23325us (23761us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.669264:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6959 00002000:00010000:0.0:1769074768.669911:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074765 -> 1769074768 00002000:00010000:0.0:1769074768.669916:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074765 -> 1769074768 00002000:00010000:0.0:1769074768.669919:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074765 -> 1769074768 00000100:00100000:0.0:1769074768.669935:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625117184:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074768.669964:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53648 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b905 expref: 25 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074768.669989:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53648 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b905 expref: 25 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074768.670001:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e000, rc 0) 00000100:00100000:0.0:1769074768.670035:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616795520:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7307us (7964us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074768.670043:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13959 00002000:00010000:3.0:1769074768.682789:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074767 -> 1769074768 00002000:00010000:3.0:1769074768.682801:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074767 -> 1769074768 00000100:00100000:3.0:1769074768.682846:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625117312:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074768.682890:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5364f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b913 expref: 23 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074768.682914:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5364f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b913 expref: 23 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074768.682933:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f600, rc 0) 00000100:00100000:3.0:1769074768.683006:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616795776:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20057us (20899us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074768.683016:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13960 00002000:00010000:2.0:1769074768.684292:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074767 -> 1769074768 00002000:00010000:2.0:1769074768.684301:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074767 -> 1769074768 00002000:00010000:2.0:1769074768.684303:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074767 -> 1769074768 00000100:00100000:2.0:1769074768.684325:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625117440:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074768.684365:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53656 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b90c expref: 25 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074768.684395:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd53656 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b90c expref: 25 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074768.684409:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47400, rc 0) 00000100:00100000:2.0:1769074768.684466:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616795648:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20402us (22335us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074768.684475:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13961 00000100:00100000:2.0:1769074768.691615:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.691659:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.691694:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.691919:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616795904 00000100:00100000:0.0:1769074768.691931:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616795904 00000100:00100000:0.0:1769074768.691979:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796160 00000100:00100000:0.0:1769074768.691982:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796160 00000100:00100000:0.0:1769074768.691997:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796032 00000100:00100000:0.0:1769074768.692000:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796032 00000100:00100000:0.0:1769074768.692018:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13962 00000100:00100000:0.0:1769074768.692027:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616795904:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074768.692047:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074768.692069:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd5365d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b91a expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074768.692129:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625117568:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074768.692245:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13963 00000100:00100000:2.0:1769074768.692255:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616796160:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074768.692265:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.692275:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53664 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b928 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074768.692310:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625117696:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074768.692401:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13964 00000100:00100000:3.0:1769074768.692413:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616796032:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074768.692421:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074768.692432:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd5366b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b921 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074768.692465:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625117824:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074768.693279:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.693408:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.698007:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.698590:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:2.0:1769074768.699397:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074767 -> 1769074768 00000100:00100000:2.0:1769074768.699435:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625117696:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074768.699494:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53664 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b928 expref: 17 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074768.699517:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53664 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b928 expref: 17 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074768.699531:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47e00, rc 0) 00000100:00100000:2.0:1769074768.699631:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616796160:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7374us (7967us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074768.699642:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13963 00002000:00010000:2.0:1769074768.701968:0:10284:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074767 -> 1769074768 00002000:00010000:2.0:1769074768.701979:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074767 -> 1769074768 00002000:00010000:2.0:1769074768.701981:0:10284:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074767 -> 1769074768 00000100:00100000:2.0:1769074768.702008:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625117824:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074768.702047:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd5366b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b921 expref: 16 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074768.702076:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd5366b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b921 expref: 16 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074768.702089:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:2.0:1769074768.702174:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616796032:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9762us (10477us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074768.702184:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13964 00000100:00100000:3.0:1769074768.702762:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796288 00000100:00100000:3.0:1769074768.702771:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796288 00002000:00100000:2.0:1769074768.703143:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54cde5880 x1855007616796288/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 504/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.703153:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde5880 with x1855007616796288 ext(0->1048575) 00002000:00010000:2.0:1769074768.703158:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde5880 00000100:00100000:2.0:1769074768.703204:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8463 00000100:00100000:2.0:1769074768.703213:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616796288:12345-192.168.204.60@tcp:3:cat.0 00002000:00010000:0.0:1769074768.715709:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074767 -> 1769074768 00002000:00010000:0.0:1769074768.715726:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074767 -> 1769074768 00002000:00010000:0.0:1769074768.715729:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074767 -> 1769074768 00000100:00100000:0.0:1769074768.715751:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625117568:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074768.715787:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd5365d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b91a expref: 14 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074768.716327:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd5365d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b91a expref: 14 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074768.716343:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876c00, rc 0) 00000100:00100000:0.0:1769074768.716557:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616795904:12345-192.168.204.60@tcp:101:cat.0 Request processed in 24529us (24962us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074768.716566:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13962 00000100:00100000:3.0:1769074768.716925:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616796288:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13712us (18338us total) trans 0 rc 1011712/1011712 00000100:00100000:3.0:1769074768.716934:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8463 00002000:00100000:3.0:1769074768.716944:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54cde5880 x1855007616796288/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 504/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1011712/1011712 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.716952:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde5880 with x1855007616796288 ext(0->1048575) 00002000:00010000:3.0:1769074768.716957:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde5880 00000100:00100000:2.0:1769074768.721148:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.721227:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796416 00000100:00100000:0.0:1769074768.721239:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796416 00000100:00100000:0.0:1769074768.721297:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13965 00000100:00100000:0.0:1769074768.721307:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62778:x1855007616796416:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074768.721325:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074768.721344:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b92f expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.721387:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.721395:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074768.721397:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074768.721402:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.721440:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 23 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.721459:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074768.721473:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625117952:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074768.721567:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074768.724290:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625117952:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074768.724348:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.724389:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877e00, rc 0) 00000100:00100000:0.0:1769074768.724479:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62778:x1855007616796416:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3169us (3338us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.724495:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13965 00000100:00100000:1.0:1769074768.727221:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.727927:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796544 00000100:00100000:3.0:1769074768.727938:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796544 00002000:00100000:2.0:1769074768.728149:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (172032->208895) req@ffff89b54cde5500 x1855007616796544/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.728161:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde5500 with x1855007616796544 ext(172032->208895) 00010000:00010000:2.0:1769074768.728171:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 24 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074768.728188:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 24 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:2.0:1769074768.729440:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cde5500 00000100:00100000:2.0:1769074768.729548:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 737 00000100:00100000:2.0:1769074768.729564:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616796544:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074768.742080:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616796544:12345-192.168.204.60@tcp:4:cat.0 Request processed in 12513us (14859us total) trans 17179874499 rc 0/0 00000100:00100000:3.0:1769074768.742095:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 737 00002000:00100000:3.0:1769074768.742107:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (172032->208895) req@ffff89b54cde5500 x1855007616796544/t17179874499(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/448 e 0 to 0 dl 1769074779 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.742118:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde5500 with x1855007616796544 ext(172032->208895) 00010000:00010000:3.0:1769074768.742129:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.742145:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:3.0:1769074768.742153:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cde5500 00000100:00100000:2.0:1769074768.742260:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.742434:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.743572:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796672 00000100:00100000:0.0:1769074768.743588:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796672 00000100:00100000:0.0:1769074768.743703:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796800 00000100:00100000:0.0:1769074768.743707:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796800 00000100:00100000:2.0:1769074768.743747:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13966 00000100:00100000:2.0:1769074768.743760:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62772:x1855007616796672:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074768.743779:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.743804:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b936 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.743874:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.743884:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.743891:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:2.0:1769074768.743893:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:2.0:1769074768.743897:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2203648->2744319] (req 2203648->2744319) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.743936:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.743954:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 22 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.743970:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625118080:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074768.744073:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.744089:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.744099:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625118208:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074768.744125:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.745349:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13967 00000100:00100000:3.0:1769074768.745384:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62775:x1855007616796800:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074768.745402:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074768.745429:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b93d expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.745457:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.745465:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074768.745466:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074768.745470:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 23 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.745501:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.745512:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.745524:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625118336:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074768.746525:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.761005:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625118080:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074768.761027:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625118208:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074768.761038:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:3.0:1769074768.761067:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2203648->2744319] (req 2203648->2744319) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.761098:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:0.0:1769074768.761126:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.761214:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797056 00000100:00100000:3.0:1769074768.761217:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62772:x1855007616796672:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17409us (18905us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.761222:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797056 00000100:00100000:3.0:1769074768.761228:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13966 00010000:00010000:1.0:1769074768.761256:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.761279:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797184 00000100:00100000:1.0:1769074768.761283:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797184 00000100:00100000:3.0:1769074768.761294:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6960 00010000:00010000:1.0:1769074768.761295:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.761305:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616797056:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1769074768.761313:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6961 00000100:00100000:1.0:1769074768.761318:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616797184:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.761320:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.761325:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.761345:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.761345:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.761358:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.761358:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.761397:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd535a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b895 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.761397:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53599 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 172032->208895) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b88e expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.761411:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074768.761411:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.761413:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074768.761421:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.761452:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.761490:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074768.761504:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625118464:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.761950:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616797184:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 629us (821us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.761958:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6961 00000100:00100000:2.0:1769074768.763045:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.763059:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625118336:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074768.763087:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 23 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.763105:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d200, rc 0) 00000100:00100000:1.0:1769074768.763142:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62775:x1855007616796800:12345-192.168.204.60@tcp:101:cat.0 Request processed in 17783us (20708us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.763150:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13967 00000100:00100000:2.0:1769074768.764026:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.764185:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616796928 00000100:00100000:1.0:1769074768.764193:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616796928 00010000:00010000:1.0:1769074768.764232:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074768.764269:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6962 00000100:00100000:1.0:1769074768.764278:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616796928:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074768.764293:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074768.764313:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.764325:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.764348:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053000/0x7fe2323adfd535b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8aa expref: 23 pid: 8398 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074768.764363:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.764364:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074768.764371:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.764390:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074768.764432:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074768.764463:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625118592:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.764521:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648c780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.764871:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b543ade3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074768.764889:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797312 00000100:00100000:3.0:1769074768.764896:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797312 00002000:00100000:3.0:1769074768.764932:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (2039808->2097151) req@ffff89b5476fa300 x1855007616797312/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.764942:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fa300 with x1855007616797312 ext(2039808->2097151) 00010000:00010000:3.0:1769074768.764948:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.764960:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:3.0:1769074768.765625:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5476fa300 00000100:00100000:3.0:1769074768.765666:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 738 00000100:00100000:3.0:1769074768.765676:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616797312:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074768.771061:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.771162:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625118464:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074768.771250:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797440 00000100:00100000:1.0:1769074768.771260:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797440 00002000:00100000:1.0:1769074768.771313:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2203648->2207743) req@ffff89b549203100 x1855007616797440/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.771324:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203100 with x1855007616797440 ext(2203648->2207743) 00002000:00010000:1.0:1769074768.771334:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549203100 00010000:00010000:3.0:1769074768.771366:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074768.771371:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8464 00000100:00100000:1.0:1769074768.771389:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797440:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074768.771439:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616797056:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10135us (10400us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.771447:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6960 00000100:00100000:1.0:1769074768.771532:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797440:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 144us (481us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074768.771538:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8464 00002000:00100000:1.0:1769074768.771544:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2203648->2207743) req@ffff89b549203100 x1855007616797440/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.771551:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203100 with x1855007616797440 ext(2203648->2207743) 00002000:00010000:1.0:1769074768.771554:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549203100 00000100:00100000:1.0:1769074768.771583:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625118592:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074768.771595:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074768.771629:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616796928:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7352us (7607us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.771637:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6962 00000100:00100000:3.0:1769074768.773385:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797312:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 7688us (10322us total) trans 21474841185 rc 0/0 00000100:00100000:3.0:1769074768.773401:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 738 00002000:00100000:3.0:1769074768.773415:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (2039808->2097151) req@ffff89b5476fa300 x1855007616797312/t21474841185(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/448 e 0 to 0 dl 1769074779 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.773426:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fa300 with x1855007616797312 ext(2039808->2097151) 00010000:00010000:3.0:1769074768.773437:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.773454:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:3.0:1769074768.773464:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5476fa300 00000100:00100000:2.0:1769074768.787531:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.787575:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.787620:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.787710:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797568 00000100:00100000:0.0:1769074768.787718:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797824 00000100:00100000:0.0:1769074768.787729:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797824 00000100:00100000:3.0:1769074768.787729:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797568 00010000:00010000:0.0:1769074768.787781:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 18 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00100000:3.0:1769074768.787793:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5476f9c00 x1855007616797568/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.787802:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9c00 with x1855007616797568 ext(0->1048575) 00002000:00010000:3.0:1769074768.787839:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f9c00 00000100:00100000:0.0:1769074768.787850:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6963 00000100:00100000:0.0:1769074768.787861:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1769074768.787862:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797696 00000100:00100000:3.0:1769074768.787865:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797696 00010000:00010000:0.0:1769074768.787881:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00100000:3.0:1769074768.787881:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2740224->2744319) req@ffff89b67a0f7b80 x1855007616797696/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.787888:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7b80 with x1855007616797696 ext(2740224->2744319) 00002000:00010000:3.0:1769074768.787891:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f7b80 00010000:00010000:0.0:1769074768.787903:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 19 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.787911:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8465 00000100:00100000:3.0:1769074768.787917:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616797568:12345-192.168.204.60@tcp:3:multiop.0 00010000:00010000:0.0:1769074768.787919:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 19 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.787948:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd53609 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2039808->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e2 expref: 19 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.787969:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.787972:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.787981:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.787992:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.788039:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074768.788055:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625118720:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074768.788142:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074768.788887:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8466 00000100:00100000:2.0:1769074768.788901:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797696:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074768.789035:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797696:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 137us (1458us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074768.795205:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625118720:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074768.795241:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.795309:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616797824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7448us (7687us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.795322:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6963 00000100:00100000:0.0:1769074768.797965:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625118848:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074768.798067:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074768.798141:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625118848 00000100:00100000:0.0:1769074768.798152:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625118848 00000100:00100000:0.0:1769074768.798207:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2645 00000100:00100000:0.0:1769074768.798215:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625118848:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074768.798254:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6438912 pending 0 free 1446150144 avail 1299910656 00000100:00100000:0.0:1769074768.798305:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625118848:12345-0@lo:41:osp-pre-1-0.0 Request processed in 90us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.798313:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2645 00000100:00100000:0.0:1769074768.798369:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625118848:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074768.798930:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074768.798940:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625118976:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074768.798971:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074768.798999:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625118976 00000100:00100000:0.0:1769074768.799004:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625118976 00000100:00100000:0.0:1769074768.799031:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2646 00000100:00100000:0.0:1769074768.799038:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625118976:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074768.799065:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6389760 pending 0 free 1445400576 avail 1299161088 00000100:00100000:0.0:1769074768.799102:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625118976:12345-0@lo:41:osp-pre-0-1.0 Request processed in 65us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.799109:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2646 00000100:00100000:0.0:1769074768.799138:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625118976:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074768.801936:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8466 00002000:00100000:2.0:1769074768.801962:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (2740224->2744319) req@ffff89b67a0f7b80 x1855007616797696/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.801979:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7b80 with x1855007616797696 ext(2740224->2744319) 00002000:00010000:2.0:1769074768.801986:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f7b80 00000100:00100000:3.0:1769074768.803246:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625119488:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.803304:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074768.803331:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625120000:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.803354:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074768.803391:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1769074768.803397:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625119360:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.803418:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074768.803436:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625119872:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.803455:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074768.803469:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625120384:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.803487:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074768.803515:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625120000 00000100:00100000:3.0:1769074768.803522:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625120000 00000100:00100000:3.0:1769074768.803572:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119872 00000100:00100000:3.0:1769074768.803576:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119872 00000100:00100000:3.0:1769074768.803595:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625120384 00000100:00100000:3.0:1769074768.803598:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625120384 00000100:00100000:3.0:1769074768.803613:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2301 00000100:00100000:3.0:1769074768.803621:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625120000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.803652:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616797568:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 15731us (16129us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074768.803663:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8465 00000100:00100000:3.0:1769074768.803663:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625120000:12345-0@lo:400:kworker.0 Request processed in 44us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.803669:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2301 00002000:00100000:0.0:1769074768.803672:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5476f9c00 x1855007616797568/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1769074768.803682:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2302 00002000:00100000:0.0:1769074768.803688:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9c00 with x1855007616797568 ext(0->1048575) 00000100:00100000:3.0:1769074768.803688:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625119872:12345-0@lo:400:kworker.0 00002000:00010000:0.0:1769074768.803693:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f9c00 00000100:00100000:3.0:1769074768.803711:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625119872:12345-0@lo:400:kworker.0 Request processed in 23us (255us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.803717:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2302 00000100:00100000:2.0:1769074768.803842:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625119232:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.803872:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074768.803884:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625119744:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.803903:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074768.803913:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625120256:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.803932:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074768.803955:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1769074768.803961:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625119104:0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.803963:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119488 00000100:00100000:0.0:1769074768.803967:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119488 00000100:00100000:2.0:1769074768.803990:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074768.803993:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119360 00000100:00100000:0.0:1769074768.803995:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119360 00000100:00100000:2.0:1769074768.804012:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625119616:0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.804016:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625120256 00000100:00100000:0.0:1769074768.804018:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625120256 00000100:00100000:2.0:1769074768.804031:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074768.804034:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1438 00000100:00100000:0.0:1769074768.804039:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625119488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.804041:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625120128:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.804059:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074768.804069:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625119488:12345-0@lo:400:kworker.0 Request processed in 30us (765us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.804075:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1438 00000100:00100000:2.0:1769074768.804083:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119104 00000100:00100000:2.0:1769074768.804087:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119104 00000100:00100000:0.0:1769074768.804096:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625120128 00000100:00100000:0.0:1769074768.804098:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625120128 00000100:00100000:2.0:1769074768.804112:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:0.0:1769074768.804114:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1439 00000100:00100000:2.0:1769074768.804118:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625119104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.804119:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625119360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.804146:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625119360:12345-0@lo:400:kworker.0 Request processed in 26us (727us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.804146:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625119104:12345-0@lo:400:kworker.0 Request processed in 29us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.804152:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1439 00000100:00100000:2.0:1769074768.804152:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:0.0:1769074768.804162:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1440 00000100:00100000:0.0:1769074768.804167:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625120256:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.804182:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1441 00000100:00100000:2.0:1769074768.804189:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625120128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.804192:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625120256:12345-0@lo:400:kworker.0 Request processed in 25us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074768.804197:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1440 00000100:00100000:2.0:1769074768.804212:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625120128:12345-0@lo:400:kworker.0 Request processed in 25us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.804217:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1441 00000100:00100000:2.0:1769074768.804248:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625119104:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.804263:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841168 gen 2 00000004:00080000:2.0:1769074768.804267:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759df80, transno 21474841153 00000004:00080000:2.0:1769074768.804287:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759b480, transno 21474841154 00000004:00080000:2.0:1769074768.804291:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759ce00, transno 21474841155 00000004:00080000:2.0:1769074768.804293:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759f480, transno 21474841156 00000004:00080000:2.0:1769074768.804296:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475de300, transno 21474841157 00000004:00080000:2.0:1769074768.804299:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759fb80, transno 21474841158 00000004:00080000:2.0:1769074768.804302:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759e680, transno 21474841159 00000004:00080000:2.0:1769074768.804304:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759d500, transno 21474841160 00000004:00080000:2.0:1769074768.804307:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5446c0e00, transno 21474841161 00000004:00080000:2.0:1769074768.804309:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5446c0000, transno 21474841162 00000004:00080000:2.0:1769074768.804311:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5446c2a00, transno 21474841163 00000004:00080000:2.0:1769074768.804314:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc000, transno 21474841164 00000004:00080000:2.0:1769074768.804317:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df800, transno 21474841165 00000004:00080000:2.0:1769074768.804321:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491edf80, transno 21474841166 00000004:00080000:2.0:1769074768.804323:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ed500, transno 21474841167 00000004:00080000:2.0:1769074768.804326:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ded80, transno 21474841168 00000100:00100000:2.0:1769074768.804331:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625120128:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.804342:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874487 gen 2 00000004:00080000:2.0:1769074768.804345:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545388380, transno 17179874470 00000004:00080000:2.0:1769074768.804353:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545388000, transno 17179874471 00000004:00080000:2.0:1769074768.804355:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545389c00, transno 17179874472 00000004:00080000:2.0:1769074768.804358:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545389f80, transno 17179874473 00000004:00080000:2.0:1769074768.804361:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545389500, transno 17179874474 00000004:00080000:2.0:1769074768.804364:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54538a300, transno 17179874475 00000004:00080000:2.0:1769074768.804367:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54538b100, transno 17179874476 00000004:00080000:2.0:1769074768.804370:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b542fa9f80, transno 17179874477 00000004:00080000:2.0:1769074768.804382:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475d9c00, transno 17179874478 00000004:00080000:2.0:1769074768.804385:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df480, transno 17179874479 00000004:00080000:2.0:1769074768.804387:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dd880, transno 17179874480 00000004:00080000:2.0:1769074768.804389:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dd500, transno 17179874481 00000004:00080000:2.0:1769074768.804391:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc700, transno 17179874482 00000004:00080000:2.0:1769074768.804394:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dca80, transno 17179874483 00000004:00080000:2.0:1769074768.804396:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddf80, transno 17179874484 00000004:00080000:2.0:1769074768.804399:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddc00, transno 17179874485 00000004:00080000:2.0:1769074768.804402:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dfb80, transno 17179874486 00000004:00080000:2.0:1769074768.804404:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5446c2d80, transno 17179874487 00000100:00100000:2.0:1769074768.804408:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625120256:0@lo:400:kworker.0 00000100:00100000:1.0:1769074768.804613:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119232 00000100:00100000:1.0:1769074768.804619:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119232 00000004:00080000:0.0:1769074768.804690:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 16 done 00000100:00100000:3.0:1769074768.804706:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119744 00000100:00100000:3.0:1769074768.804709:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119744 00000004:00080000:0.0:1769074768.804854:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 18 done 00000100:00100000:1.0:1769074768.808617:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625119616 00000100:00100000:1.0:1769074768.808646:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625119616 00000100:00100000:1.0:1769074768.808747:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2303 00000100:00100000:1.0:1769074768.808762:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625120384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074768.808944:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625120384:12345-0@lo:400:kworker.0 Request processed in 184us (5454us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.808954:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2303 00000100:00100000:1.0:1769074768.808982:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2304 00000100:00100000:1.0:1769074768.808989:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625119232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074768.809049:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625119232:12345-0@lo:400:kworker.0 Request processed in 60us (5176us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.809056:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2304 00000100:00100000:1.0:1769074768.809073:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2305 00000100:00100000:1.0:1769074768.809090:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625119616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074768.809120:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625119616:12345-0@lo:400:kworker.0 Request processed in 41us (5088us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074768.809126:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2305 00000100:00100000:2.0:1769074768.809769:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625119616:0@lo:400:kworker.0 00000100:00100000:2.0:1769074768.809789:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625119232:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.811624:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2306 00000100:00100000:3.0:1769074768.811648:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625119744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.811762:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625119744:12345-0@lo:400:kworker.0 Request processed in 127us (7857us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.811769:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2306 00000100:00080000:3.0:1769074768.811824:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074768.811829:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074768.811833:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074768.811835:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074768.811889:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625119360:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.811905:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625119872:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.811919:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5446c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625120384:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.811934:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625119488:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.811944:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625120000:0@lo:400:kworker.0 00000100:00100000:3.0:1769074768.812106:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.812620:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625119744:0@lo:400:kworker.0 00000100:00100000:0.0:1769074768.813890:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616797952 00000100:00100000:0.0:1769074768.813910:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616797952 00002000:00100000:0.0:1769074768.813977:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546f93100 x1855007616797952/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.813989:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93100 with x1855007616797952 ext(1048576->2097151) 00002000:00010000:0.0:1769074768.813996:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f93100 00000100:00100000:0.0:1769074768.814039:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8467 00000100:00100000:0.0:1769074768.814051:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616797952:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074768.844333:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616797952:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30257us (32203us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074768.844354:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8467 00002000:00100000:0.0:1769074768.844410:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b546f93100 x1855007616797952/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.844422:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f93100 with x1855007616797952 ext(1048576->2097151) 00002000:00010000:0.0:1769074768.844431:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f93100 00000100:00100000:1.0:1769074768.851142:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.851267:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.851470:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616798080 00000100:00100000:2.0:1769074768.851483:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616798080 00000100:00100000:2.0:1769074768.851546:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616798208 00000100:00100000:2.0:1769074768.851550:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616798208 00000100:00100000:2.0:1769074768.851586:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13968 00000100:00100000:2.0:1769074768.851599:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62778:x1855007616798080:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074768.851625:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.851647:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b944 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851683:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.851686:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.851696:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851714:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851732:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47400, rc 0) 00000100:00100000:2.0:1769074768.851791:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62778:x1855007616798080:12345-192.168.204.60@tcp:101:cat.0 Request processed in 195us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.851799:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13968 00000100:00100000:2.0:1769074768.851853:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13969 00000100:00100000:2.0:1769074768.851860:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62775:x1855007616798208:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074768.851867:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.851878:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b94b expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851897:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851904:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074768.851906:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074768.851908:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 26 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851946:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.851959:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 13 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.851972:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625120512:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074768.852025:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.861951:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.862058:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625120512:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074768.862092:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 26 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.862114:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47600, rc 0) 00000100:00100000:2.0:1769074768.862176:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62775:x1855007616798208:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10315us (10909us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.862185:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13969 00000100:00100000:0.0:1769074768.862323:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616798336 00000100:00100000:0.0:1769074768.862331:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616798336 00002000:00100000:0.0:1769074768.862389:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1429503) req@ffff89b546c69f80 x1855007616798336/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.862399:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69f80 with x1855007616798336 ext(1048576->1429503) 00010000:00010000:0.0:1769074768.862407:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 14 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.862421:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 14 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:0.0:1769074768.862429:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546c69f80 00000100:00100000:0.0:1769074768.862467:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 739 00000100:00100000:0.0:1769074768.862476:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616798336:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074768.867927:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.867965:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.868133:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616798720 00000100:00100000:2.0:1769074768.868143:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616798720 00002000:00100000:2.0:1769074768.868189:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67a0f7800 x1855007616798720/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.868199:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7800 with x1855007616798720 ext(1048576->2097151) 00002000:00010000:2.0:1769074768.868205:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f7800 00000100:00100000:2.0:1769074768.868229:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8468 00000100:00100000:2.0:1769074768.868237:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616798720:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074768.868920:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616798592 00000100:00100000:0.0:1769074768.868928:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616798592 00000100:00100000:0.0:1769074768.868977:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13308 00000100:00100000:0.0:1769074768.868986:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62781:x1855007616798592:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074768.869004:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074768.869024:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53695 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b952 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.869093:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869107:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869119:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869135:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869143:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869149:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869293:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd536aa lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869305:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd536aa lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869313:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd536aa lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074768.869324:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074768.869437:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869522:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869528:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869535:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869542:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074768.869546:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869553:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869566:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074768.869569:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074768.869573:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869583:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5369c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869598:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869604:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869610:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869615:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074768.869618:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869623:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869629:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47600) 00010000:00010000:0.0:1769074768.869632:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074768.869634:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869639:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd536a3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074768.869681:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53695 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b952 expref: 230 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.869695:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd536aa lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b952 expref: 229 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.869705:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074768.869708:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074768.869711:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877400, rc 0) 00000100:00100000:0.0:1769074768.869779:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62781:x1855007616798592:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 794us (1864us total) trans 8589943923 rc 0/0 00000100:00100000:0.0:1769074768.869787:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13308 00000100:00100000:0.0:1769074768.876632:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616798336:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 14146us (14679us total) trans 17179874500 rc 0/0 00000100:00100000:0.0:1769074768.876647:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 739 00002000:00100000:0.0:1769074768.876662:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->1429503) req@ffff89b546c69f80 x1855007616798336/t17179874500(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/448 e 0 to 0 dl 1769074779 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.876672:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c69f80 with x1855007616798336 ext(1048576->1429503) 00010000:00010000:0.0:1769074768.876681:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.876699:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:0.0:1769074768.876708:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546c69f80 00000100:00100000:0.0:1769074768.878045:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.878087:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.878171:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799104 00000100:00100000:1.0:1769074768.878183:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799104 00000100:00100000:3.0:1769074768.878295:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799232 00000100:00100000:3.0:1769074768.878301:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799232 00000100:00100000:3.0:1769074768.878348:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 998 00000100:00100000:3.0:1769074768.878358:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:2406:x1855007616799232:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.878386:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.878412:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd536aa lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b952 expref: 230 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074768.878429:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074768.878435:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.878504:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:2406:x1855007616799232:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 147us (416us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.878511:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 998 00000100:00100000:1.0:1769074768.878567:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4656 00000100:00100000:1.0:1769074768.878574:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:62781:x1855007616799104:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074768.878779:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62781:x1855007616799104:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 205us (741us total) trans 8589943924 rc 0/0 00000100:00100000:1.0:1769074768.878786:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4656 00000100:00100000:3.0:1769074768.883089:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.883226:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616798848 00000100:00100000:3.0:1769074768.883239:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616798848 00002000:00100000:3.0:1769074768.883294:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1429504->2097151) req@ffff89b546f92a00 x1855007616798848/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.883306:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92a00 with x1855007616798848 ext(1429504->2097151) 00010000:00010000:3.0:1769074768.883317:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.883335:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:3.0:1769074768.883342:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f92a00 00000100:00100000:3.0:1769074768.883390:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 740 00000100:00100000:3.0:1769074768.883399:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616798848:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074768.891045:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616798720:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 22793us (23065us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074768.891061:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8468 00002000:00100000:2.0:1769074768.891075:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67a0f7800 x1855007616798720/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.891086:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7800 with x1855007616798720 ext(1048576->2097151) 00002000:00010000:2.0:1769074768.891093:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f7800 00000100:00100000:3.0:1769074768.893174:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616798848:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 9763us (10089us total) trans 0 rc 667648/667648 00000100:00100000:3.0:1769074768.893192:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 740 00002000:00100000:3.0:1769074768.893207:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1429504->2097151) req@ffff89b546f92a00 x1855007616798848/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:H/600/0 rc 667648/667648 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.893217:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92a00 with x1855007616798848 ext(1429504->2097151) 00010000:00010000:3.0:1769074768.893229:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.893246:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 16 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:3.0:1769074768.893259:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f92a00 00000100:00100000:3.0:1769074768.893980:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.894402:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799360 00000100:00100000:2.0:1769074768.894415:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799360 00000100:00100000:2.0:1769074768.894555:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13970 00000100:00100000:2.0:1769074768.894565:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62781:x1855007616799360:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074768.894582:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.894604:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b959 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.894637:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 23 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.894647:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.894653:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.894656:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.894661:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1380352->1490943] (req 1380352->1490943) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.894700:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.894712:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 17 pid: 8396 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.894725:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625120640:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074768.894853:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 23 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.894863:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 23 pid: 10284 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.894872:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625120768:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074768.894901:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.898702:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.898953:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799488 00000100:00100000:3.0:1769074768.898963:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799488 00010000:00010000:3.0:1769074768.899007:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 24 pid: 10284 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074768.899054:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6964 00000100:00100000:3.0:1769074768.899065:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616799488:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074768.899084:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074768.899110:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 24 pid: 10284 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.899122:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 24 pid: 10284 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074768.899153:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.899216:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616799488:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 152us (524us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.899223:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6964 00000100:00100000:2.0:1769074768.901126:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625120640:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074768.901154:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53680 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b93d expref: 23 pid: 10284 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.901173:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625120768:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074768.901223:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1380352->1490943] (req 1380352->1490943) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.901257:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46600, rc 0) 00000100:00100000:2.0:1769074768.901369:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62781:x1855007616799360:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6803us (7393us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.901386:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13970 00000100:00100000:1.0:1769074768.912182:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.912299:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074768.912587:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074768.913801:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799872 00000100:00100000:2.0:1769074768.913850:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799872 00000100:00100000:2.0:1769074768.913929:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13971 00000100:00100000:2.0:1769074768.913938:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62778:x1855007616799872:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074768.913957:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074768.913985:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b960 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.914018:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074768.914024:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.914034:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.914046:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46800, rc 0) 00000100:00100000:2.0:1769074768.914104:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62778:x1855007616799872:12345-192.168.204.60@tcp:101:cat.0 Request processed in 166us (1516us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074768.914110:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13971 00000100:00100000:0.0:1769074768.915782:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799616 00000100:00100000:0.0:1769074768.915802:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799616 00010000:00010000:0.0:1769074768.915863:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074768.915897:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616799744 00000100:00100000:0.0:1769074768.915900:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616799744 00000100:00100000:2.0:1769074768.915910:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6965 00010000:00010000:0.0:1769074768.915914:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 18 pid: 8396 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074768.915921:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616799616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1769074768.915934:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6966 00010000:00010000:2.0:1769074768.915935:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1769074768.915939:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616799744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074768.915946:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074768.915956:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.915957:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 18 pid: 8396 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074768.915969:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.915981:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 18 pid: 8396 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074768.915992:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f71400/0x7fe2323adfd53610 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->1429503) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8e9 expref: 17 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.915993:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47400/0x7fe2323adfd53687 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b944 expref: 18 pid: 8396 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074768.916007:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.916007:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.916009:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.916009:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074768.916016:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.916016:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 25 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.916028:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.916028:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 25 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.916036:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.916037:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074768.916040:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.916047:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074768.916064:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074768.916065:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074768.916088:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625121024:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074768.916089:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625120896:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074768.917676:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00010000:00010000:2.0:1769074768.918289:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 25 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074768.918305:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625121152:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074768.918328:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074768.929272:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074768.929272:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625121024:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074768.929288:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625121152:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074768.929319:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074768.929411:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616799616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13491us (17239us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.929424:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6965 00000100:00100000:3.0:1769074768.929513:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800000 00000100:00100000:3.0:1769074768.929521:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800000 00002000:00100000:3.0:1769074768.929586:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5497a8a80 x1855007616800000/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.929597:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8a80 with x1855007616800000 ext(1048576->2097151) 00000100:00100000:0.0:1769074768.929606:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625120896:192.168.204.60@tcp:105: 00002000:00010000:3.0:1769074768.929616:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a8a80 00010000:00010000:0.0:1769074768.929622:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074768.929649:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616799744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13709us (17349us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074768.929652:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8469 00000100:00100000:0.0:1769074768.929657:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6966 00000100:00100000:3.0:1769074768.929661:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616800000:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074768.935529:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.935565:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074768.935619:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800128 00000100:00100000:0.0:1769074768.935629:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800128 00002000:00100000:0.0:1769074768.935678:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b549202680 x1855007616800128/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.935690:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549202680 with x1855007616800128 ext(1048576->2097151) 00002000:00010000:0.0:1769074768.935696:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549202680 00000100:00100000:0.0:1769074768.935723:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800256 00000100:00100000:0.0:1769074768.935727:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800256 00002000:00100000:0.0:1769074768.935744:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1486848->1490943) req@ffff89b549203b80 x1855007616800256/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/0 e 0 to 0 dl 1769074779 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074768.935752:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203b80 with x1855007616800256 ext(1486848->1490943) 00002000:00010000:0.0:1769074768.935756:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549203b80 00000100:00100000:0.0:1769074768.935780:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8470 00000100:00100000:0.0:1769074768.935790:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616800128:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074768.935848:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8471 00000100:00100000:1.0:1769074768.935857:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616800256:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074768.935963:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616800256:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 107us (396us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074768.935984:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8471 00002000:00100000:1.0:1769074768.935991:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1486848->1490943) req@ffff89b549203b80 x1855007616800256/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/4536 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074768.935999:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203b80 with x1855007616800256 ext(1486848->1490943) 00002000:00010000:1.0:1769074768.936003:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549203b80 00000100:00100000:3.0:1769074768.959414:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616800000:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 29734us (30133us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074768.959434:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8469 00002000:00100000:3.0:1769074768.959451:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5497a8a80 x1855007616800000/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074768.959464:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8a80 with x1855007616800000 ext(1048576->2097151) 00002000:00010000:3.0:1769074768.959472:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497a8a80 00000100:00100000:2.0:1769074768.988720:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616800128:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 52919us (53189us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074768.988735:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8470 00002000:00100000:2.0:1769074768.988757:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b549202680 x1855007616800128/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:304/0 lens 488/440 e 0 to 0 dl 1769074779 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074768.988769:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549202680 with x1855007616800128 ext(1048576->2097151) 00002000:00010000:2.0:1769074768.988776:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549202680 00000100:00100000:1.0:1769074769.008865:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.008933:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800384 00000100:00100000:3.0:1769074769.008944:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800384 00000100:00100000:3.0:1769074769.009017:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13972 00000100:00100000:3.0:1769074769.009028:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616800384:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.009047:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.009066:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd536c6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b967 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.009137:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625121280:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074769.009241:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00080000:0.0:1769074769.009734:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074769.012598:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.012653:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.012727:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800512 00000100:00100000:0.0:1769074769.012738:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800512 00000100:00100000:2.0:1769074769.012753:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800640 00000100:00100000:2.0:1769074769.012757:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800640 00000100:00080000:3.0:1769074769.012833:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074769.012955:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13973 00000100:00100000:0.0:1769074769.012967:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616800512:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074769.012995:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074769.013015:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd536cd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b96e expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074769.013088:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625121408:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074769.013167:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.020300:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625121280:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074769.020361:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd536c6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b967 expref: 19 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.020396:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd536c6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b967 expref: 19 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.020413:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:0.0:1769074769.020482:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616800384:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11455us (11628us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074769.020491:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13972 00000100:00100000:0.0:1769074769.020527:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13974 00000100:00100000:0.0:1769074769.020539:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616800640:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074769.020553:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074769.020568:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd536d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b975 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074769.020631:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625121536:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074769.020684:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.025054:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.025227:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625121408:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074769.025273:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd536cd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b96e expref: 17 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.025298:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd536cd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b96e expref: 17 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.025310:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877000, rc 0) 00000100:00100000:0.0:1769074769.025379:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616800512:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12404us (12782us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074769.025388:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13973 00000100:00100000:2.0:1769074769.025636:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800768 00000100:00100000:2.0:1769074769.025644:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800768 00002000:00100000:2.0:1769074769.025705:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54cde7800 x1855007616800768/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.025792:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7800 with x1855007616800768 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.025797:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde7800 00000100:00100000:2.0:1769074769.025845:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8472 00000100:00100000:2.0:1769074769.025855:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616800768:12345-192.168.204.60@tcp:3:cat.0 00000100:00080000:3.0:1769074769.026595:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074769.040484:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625121536:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074769.040527:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd536d4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b975 expref: 18 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.040550:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd536d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b975 expref: 18 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.040563:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:3.0:1769074769.040633:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616800640:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20097us (27978us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074769.040642:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13974 00000100:00100000:2.0:1769074769.043322:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616800768:12345-192.168.204.60@tcp:3:cat.0 Request processed in 17455us (18265us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074769.043337:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8472 00002000:00100000:2.0:1769074769.043350:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b54cde7800 x1855007616800768/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.043361:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde7800 with x1855007616800768 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.043378:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cde7800 00000100:00100000:1.0:1769074769.043401:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.043569:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616800896 00000100:00100000:3.0:1769074769.043578:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616800896 00000100:00100000:3.0:1769074769.043637:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13975 00000100:00100000:3.0:1769074769.043647:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62778:x1855007616800896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.043662:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.043680:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b97c expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.043719:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.043727:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074769.043729:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074769.043732:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.043765:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 25 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.043777:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 25 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:3.0:1769074769.043789:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625121664:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074769.043896:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.048589:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625121664:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074769.048632:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.048653:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8800, rc 0) 00000100:00100000:3.0:1769074769.048878:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62778:x1855007616800896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5231us (5479us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.048888:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13975 00000100:00100000:1.0:1769074769.055998:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.056046:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.056127:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801152 00000100:00100000:0.0:1769074769.056139:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801152 00000100:00100000:0.0:1769074769.056206:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13976 00000100:00100000:0.0:1769074769.056217:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62775:x1855007616801152:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074769.056235:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074769.056254:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b983 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.056284:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.056298:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074769.056300:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074769.056304:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 23 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.056338:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.056350:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 16 pid: 9048 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074769.056362:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625121792:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074769.056453:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.056765:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801024 00000100:00100000:3.0:1769074769.056771:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801024 00002000:00100000:3.0:1769074769.056820:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4239359) req@ffff89b5497aa300 x1855007616801024/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074769.056831:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aa300 with x1855007616801024 ext(4194304->4239359) 00010000:00010000:3.0:1769074769.056839:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:3.0:1769074769.056850:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 26 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:3.0:1769074769.056857:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497aa300 00000100:00100000:3.0:1769074769.056883:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 741 00000100:00100000:3.0:1769074769.056890:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616801024:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074769.062126:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.062176:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.063005:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625121792:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074769.063039:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 23 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.063059:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877800, rc 0) 00000100:00100000:3.0:1769074769.063115:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62775:x1855007616801152:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6900us (7068us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.063125:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13976 00000100:00100000:3.0:1769074769.063150:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801280 00000100:00100000:3.0:1769074769.063156:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801280 00000100:00100000:0.0:1769074769.063318:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801408 00000100:00100000:0.0:1769074769.063323:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801408 00002000:00100000:0.0:1769074769.063352:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2203648->2744319) req@ffff89b549203800 x1855007616801408/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.063362:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203800 with x1855007616801408 ext(2203648->2744319) 00010000:00010000:0.0:1769074769.063380:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 18 pid: 9048 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074769.063392:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 18 pid: 9048 timeout: 1856 lvb_type: 0 00002000:00010000:0.0:1769074769.063406:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549203800 00000100:00100000:0.0:1769074769.063422:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 742 00000100:00100000:0.0:1769074769.063431:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616801408:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074769.064253:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616801024:12345-192.168.204.60@tcp:4:cat.0 Request processed in 7359us (8264us total) trans 17179874501 rc 0/0 00000100:00100000:2.0:1769074769.064262:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 741 00002000:00100000:2.0:1769074769.064269:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4239359) req@ffff89b5497aa300 x1855007616801024/t17179874501(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/448 e 0 to 0 dl 1769074780 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.064279:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aa300 with x1855007616801024 ext(4194304->4239359) 00010000:00010000:2.0:1769074769.064286:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 28 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074769.064297:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 28 pid: 28955 timeout: 1856 lvb_type: 0 00002000:00010000:2.0:1769074769.064304:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497aa300 00000100:00100000:1.0:1769074769.065348:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13977 00000100:00100000:1.0:1769074769.065365:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62784:x1855007616801280:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074769.065393:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.065414:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b98a expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.065443:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.065452:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.065457:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.065459:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.065463:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1134592->1605631] (req 1134592->1605631) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 20 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.065497:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 20 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.065508:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 20 pid: 27844 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074769.065521:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625121920:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074769.065597:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.065607:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 22 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074769.065616:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625122048:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074769.065639:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648cb40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.070169:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616801408:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 6723us (7976us total) trans 21474841186 rc 0/0 00000100:00100000:2.0:1769074769.070186:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 742 00002000:00100000:2.0:1769074769.070202:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2203648->2744319) req@ffff89b549203800 x1855007616801408/t21474841186(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/448 e 0 to 0 dl 1769074780 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.070213:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203800 with x1855007616801408 ext(2203648->2744319) 00010000:00010000:2.0:1769074769.070224:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 21 pid: 9048 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074769.070241:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 21 pid: 9048 timeout: 1856 lvb_type: 0 00002000:00010000:2.0:1769074769.070249:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549203800 00000100:00100000:0.0:1769074769.076578:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.076617:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.076667:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.076696:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801664 00000100:00100000:0.0:1769074769.076705:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801664 00010000:00010000:0.0:1769074769.076752:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074769.076784:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801536 00000100:00100000:0.0:1769074769.076787:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801536 00010000:00010000:0.0:1769074769.076801:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 28 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074769.076837:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801792 00000100:00100000:0.0:1769074769.076840:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801792 00010000:00010000:0.0:1769074769.076853:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 21 pid: 27844 timeout: 1856 lvb_type: 0 00000100:00100000:0.0:1769074769.076870:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6967 00000100:00100000:0.0:1769074769.076878:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616801664:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074769.076891:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074769.076912:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074769.076923:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 23 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074769.076948:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074769.076992:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616801664:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 114us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074769.076999:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6967 00000100:00100000:0.0:1769074769.077016:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6968 00000100:00100000:0.0:1769074769.077022:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616801536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074769.077030:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074769.077042:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 28 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074769.077052:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 28 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074769.077066:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5361e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4239359) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f7 expref: 28 pid: 28955 timeout: 1856 lvb_type: 0 00010000:00010000:0.0:1769074769.077078:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074769.077081:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074769.077088:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.077098:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.077132:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074769.077146:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625122176:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074769.077304:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6969 00000100:00100000:1.0:1769074769.077314:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616801792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074769.077322:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074769.077331:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 21 pid: 27844 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074769.077340:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 21 pid: 27844 timeout: 1856 lvb_type: 0 00010000:00010000:1.0:1769074769.077352:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.077354:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.077359:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.077365:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.077399:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.077407:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625122304:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074769.077468:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd533e7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b768 expref: 21 pid: 27844 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074769.077480:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625121920:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074769.077492:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd533d9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b75a expref: 22 pid: 28955 timeout: 1856 lvb_type: 0 00000100:00100000:1.0:1769074769.077501:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625122048:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074769.077529:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 20 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.077539:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:1.0:1769074769.077563:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62784:x1855007616801280:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12205us (15450us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.077569:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13977 00000100:00100000:0.0:1769074769.077679:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074769.078151:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6464a8e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.082569:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.082607:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.082703:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625122176:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074769.082739:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074769.082833:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616801536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5782us (6186us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.082846:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6968 00000100:00100000:2.0:1769074769.082876:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802048 00000100:00100000:2.0:1769074769.082883:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802048 00010000:00010000:2.0:1769074769.082914:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 20 pid: 9048 timeout: 1856 lvb_type: 0 00000100:00100000:2.0:1769074769.082943:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6970 00000100:00100000:2.0:1769074769.082951:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616802048:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074769.082962:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074769.082985:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 20 pid: 9048 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074769.082998:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 20 pid: 9048 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074769.083021:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53679 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2203648->3481599] (req 2203648->2744319) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b936 expref: 20 pid: 9048 timeout: 1856 lvb_type: 0 00010000:00010000:2.0:1769074769.083036:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.083038:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.083045:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.083054:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.083087:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.083100:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625122432:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074769.083148:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.083171:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616801920 00000100:00100000:2.0:1769074769.083175:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616801920 00002000:00100000:2.0:1769074769.083219:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1134592->1138687) req@ffff89b546f90e00 x1855007616801920/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.083229:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90e00 with x1855007616801920 ext(1134592->1138687) 00002000:00010000:2.0:1769074769.083234:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f90e00 00000100:00100000:2.0:1769074769.083254:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8473 00000100:00100000:2.0:1769074769.083261:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616801920:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074769.083405:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616801920:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 145us (797us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074769.083412:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8473 00002000:00100000:2.0:1769074769.083418:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1134592->1138687) req@ffff89b546f90e00 x1855007616801920/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/4536 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.083425:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f90e00 with x1855007616801920 ext(1134592->1138687) 00002000:00010000:2.0:1769074769.083428:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f90e00 00000100:00100000:1.0:1769074769.083678:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625122304:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074769.083691:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.083725:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616801792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6413us (7057us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.083732:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6969 00000100:00100000:0.0:1769074769.090197:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.090400:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802176 00000100:00100000:2.0:1769074769.090412:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802176 00002000:00100000:2.0:1769074769.090465:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5475da300 x1855007616802176/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.090475:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475da300 with x1855007616802176 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.090481:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475da300 00000100:00100000:2.0:1769074769.090520:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8474 00000100:00100000:2.0:1769074769.090530:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616802176:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074769.091488:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625122432:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074769.091511:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.091559:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616802048:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8606us (8998us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.091568:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6970 00000100:00100000:0.0:1769074769.103983:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.104090:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802304 00000100:00100000:2.0:1769074769.104101:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802304 00002000:00100000:2.0:1769074769.104151:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b5475d8e00 x1855007616802304/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.104161:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8e00 with x1855007616802304 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.104167:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475d8e00 00000100:00100000:2.0:1769074769.104211:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8475 00000100:00100000:2.0:1769074769.104221:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616802304:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074769.113362:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.115569:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2407:x1855007616802176:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 25025us (25379us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074769.115586:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8474 00002000:00100000:1.0:1769074769.115601:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5475da300 x1855007616802176/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.115611:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475da300 with x1855007616802176 ext(2097152->3145727) 00002000:00010000:1.0:1769074769.115618:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5475da300 00000100:00100000:1.0:1769074769.115646:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802432 00000100:00100000:1.0:1769074769.115653:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802432 00002000:00100000:1.0:1769074769.115688:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1601536->1605631) req@ffff89b67a0f7100 x1855007616802432/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.115696:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7100 with x1855007616802432 ext(1601536->1605631) 00002000:00010000:1.0:1769074769.115700:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f7100 00000100:00100000:0.0:1769074769.118762:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8476 00000100:00100000:0.0:1769074769.118785:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616802432:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074769.119040:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616802432:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 263us (5688us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074769.119049:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8476 00002000:00100000:0.0:1769074769.119059:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1601536->1605631) req@ffff89b67a0f7100 x1855007616802432/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/4536 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.119068:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7100 with x1855007616802432 ext(1601536->1605631) 00002000:00010000:0.0:1769074769.119074:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f7100 00000100:00100000:2.0:1769074769.123364:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616802304:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19128us (19376us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074769.123390:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8475 00002000:00100000:2.0:1769074769.123406:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b5475d8e00 x1855007616802304/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.123416:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5475d8e00 with x1855007616802304 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.123423:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5475d8e00 00000100:00100000:3.0:1769074769.132381:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.132450:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802560 00000100:00100000:3.0:1769074769.132460:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802560 00000100:00100000:3.0:1769074769.132509:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13978 00000100:00100000:3.0:1769074769.132518:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62778:x1855007616802560:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.132536:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.132554:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b991 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.132580:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074769.132582:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074769.132590:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 19 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.132605:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 19 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.132616:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9400, rc 0) 00000100:00100000:3.0:1769074769.132667:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62778:x1855007616802560:12345-192.168.204.60@tcp:101:cat.0 Request processed in 150us (306us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.132675:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13978 00000100:00100000:3.0:1769074769.146965:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.147014:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802688 00000100:00100000:1.0:1769074769.147027:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802688 00000100:00100000:1.0:1769074769.147099:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13979 00000100:00100000:1.0:1769074769.147110:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62775:x1855007616802688:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.147134:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.147160:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b998 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.147195:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.147198:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.147208:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 28 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.147240:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 28 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.147256:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876600, rc 0) 00000100:00100000:1.0:1769074769.147332:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62775:x1855007616802688:12345-192.168.204.60@tcp:101:cat.0 Request processed in 224us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.147340:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13979 00000100:00100000:3.0:1769074769.152343:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.152665:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802816 00000100:00100000:2.0:1769074769.152680:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802816 00002000:00100000:2.0:1769074769.152752:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54c846d80 x1855007616802816/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.152763:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c846d80 with x1855007616802816 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.152769:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c846d80 00000100:00100000:2.0:1769074769.152795:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8477 00000100:00100000:2.0:1769074769.152829:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616802816:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074769.160560:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.162180:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616802944 00000100:00100000:2.0:1769074769.162206:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616802944 00002000:00100000:2.0:1769074769.162266:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54c846300 x1855007616802944/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.162278:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c846300 with x1855007616802944 ext(3145728->4194303) 00002000:00010000:2.0:1769074769.162285:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c846300 00000100:00100000:2.0:1769074769.162328:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8478 00000100:00100000:2.0:1769074769.162340:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616802944:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074769.177232:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616802816:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 24416us (24889us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074769.177250:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8477 00002000:00100000:2.0:1769074769.177263:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54c846d80 x1855007616802816/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.177274:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c846d80 with x1855007616802816 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.177281:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c846d80 00000100:00100000:2.0:1769074769.184912:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616802944:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 22401us (24190us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074769.184927:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8478 00002000:00100000:2.0:1769074769.184942:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54c846300 x1855007616802944/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.184953:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c846300 with x1855007616802944 ext(3145728->4194303) 00002000:00010000:2.0:1769074769.184959:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c846300 00000100:00100000:3.0:1769074769.202252:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.202959:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616803072 00000100:00100000:2.0:1769074769.202979:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616803072 00002000:00100000:2.0:1769074769.203037:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54c845180 x1855007616803072/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.203050:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845180 with x1855007616803072 ext(3145728->4194303) 00002000:00010000:2.0:1769074769.203056:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c845180 00000100:00100000:2.0:1769074769.203411:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8479 00000100:00100000:2.0:1769074769.203423:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616803072:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074769.232878:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.232977:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616803200 00000100:00100000:1.0:1769074769.232989:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616803200 00000100:00100000:1.0:1769074769.233049:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13980 00000100:00100000:1.0:1769074769.233058:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62778:x1855007616803200:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.233079:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.233097:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b99f expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.233112:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616803072:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29692us (30872us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074769.233120:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8479 00010000:00010000:1.0:1769074769.233125:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00002000:00100000:2.0:1769074769.233130:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54c845180 x1855007616803072/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00010000:00010000:1.0:1769074769.233133:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074769.233135:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074769.233139:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00002000:00100000:2.0:1769074769.233139:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845180 with x1855007616803072 ext(3145728->4194303) 00002000:00010000:2.0:1769074769.233144:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c845180 00010000:00010000:1.0:1769074769.233169:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.233182:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 21 pid: 28955 timeout: 1857 lvb_type: 0 00000100:00100000:1.0:1769074769.233196:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625122560:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074769.233250:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648cb40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.235319:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.235647:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616803456 00000100:00100000:1.0:1769074769.235656:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616803456 00000100:00100000:1.0:1769074769.235728:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13309 00000100:00100000:1.0:1769074769.235737:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62788:x1855007616803456:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074769.235761:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.235782:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53705 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9a6 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.235892:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.235906:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.235919:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.235934:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.235941:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.235947:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236056:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5371a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236068:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5371a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236074:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5371a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074769.236087:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074769.236167:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236174:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236179:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236185:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236191:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074769.236194:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236201:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236213:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074769.236216:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074769.236220:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236226:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5370c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236238:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236243:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236251:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236255:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074769.236258:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236262:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236268:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615200) 00010000:00010000:1.0:1769074769.236270:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074769.236272:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236277:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877400/0x7fe2323adfd53713 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.236321:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53705 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9a6 expref: 232 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.236332:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5371a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9a6 expref: 231 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.236340:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074769.236342:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074769.236345:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:1.0:1769074769.236429:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62788:x1855007616803456:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 691us (1116us total) trans 8589943925 rc 0/0 00000100:00100000:1.0:1769074769.236447:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13309 00000100:00100000:1.0:1769074769.245016:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625122560:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074769.245063:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.245083:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877000, rc 0) 00000100:00100000:1.0:1769074769.245150:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62778:x1855007616803200:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12091us (12282us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.245160:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13980 00000100:00100000:3.0:1769074769.250286:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.250334:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.250465:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616803584 00000100:00100000:2.0:1769074769.250480:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616803584 00002000:00100000:2.0:1769074769.250546:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3481600->4194303) req@ffff89b54c847100 x1855007616803584/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.250558:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c847100 with x1855007616803584 ext(3481600->4194303) 00010000:00010000:2.0:1769074769.250567:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 22 pid: 28955 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.250581:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 22 pid: 28955 timeout: 1857 lvb_type: 0 00002000:00010000:2.0:1769074769.250589:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54c847100 00000100:00100000:2.0:1769074769.250630:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 743 00000100:00100000:2.0:1769074769.250640:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616803584:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074769.251447:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616803712 00000100:00100000:1.0:1769074769.251454:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616803712 00000100:00100000:1.0:1769074769.251498:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13981 00000100:00100000:1.0:1769074769.251505:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62775:x1855007616803712:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.251517:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.251535:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9ad expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.251558:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074769.251563:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.251574:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.251585:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:1.0:1769074769.251637:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62775:x1855007616803712:12345-192.168.204.60@tcp:101:cat.0 Request processed in 132us (1302us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.251645:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13981 00000100:00100000:3.0:1769074769.260394:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.260454:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.260574:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804096 00000100:00100000:2.0:1769074769.260585:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804096 00000100:00100000:2.0:1769074769.260646:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4657 00000100:00100000:2.0:1769074769.260655:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62788:x1855007616804096:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074769.260932:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+232:62788:x1855007616804096:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 277us (477us total) trans 8589943926 rc 0/0 00000100:00100000:2.0:1769074769.260940:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4657 00000100:00100000:1.0:1769074769.261474:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616803840 00000100:00100000:1.0:1769074769.261483:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616803840 00000100:00100000:1.0:1769074769.261531:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 999 00000100:00100000:1.0:1769074769.261538:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+232:2409:x1855007616803840:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074769.261550:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074769.261574:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5371a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9a6 expref: 232 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.261593:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074769.261598:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.261636:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:2409:x1855007616803840:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 98us (1252us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.261642:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 999 00000100:00100000:0.0:1769074769.309399:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074769.309426:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625122688:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074769.309505:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074769.309560:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625122688 00000100:00100000:0.0:1769074769.309569:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625122688 00000100:00100000:0.0:1769074769.309616:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1160 00000100:00100000:0.0:1769074769.309624:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625122688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074769.309650:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1060980032 pending 2543616 free 3900067840 avail 3668545536 00000100:00100000:0.0:1769074769.309709:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625122688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074769.309729:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1160 00000100:00100000:0.0:1769074769.309782:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625122688:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1769074769.309978:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074769.310134:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625122816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074769.310172:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074769.310339:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625122816 00000100:00100000:2.0:1769074769.310344:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625122816 00000100:00100000:2.0:1769074769.310390:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1161 00000100:00100000:2.0:1769074769.310398:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625122816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074769.310412:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 630784 granted 1090311488 pending 3538944 free 3905998848 avail 3674476544 00000100:00100000:2.0:1769074769.310454:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625122816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.310461:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1161 00000100:00100000:1.0:1769074769.310538:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625122816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074769.310862:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074769.310874:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625122944:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074769.310906:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074769.311028:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625122944 00000100:00100000:2.0:1769074769.311033:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625122944 00000100:00100000:2.0:1769074769.311067:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1162 00000100:00100000:2.0:1769074769.311074:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625122944:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074769.311086:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 630784 granted 1090311488 pending 3538944 free 3905998848 avail 3674476544 00000100:00100000:2.0:1769074769.311126:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625122944:12345-0@lo:13:osp-pre-0-1.0 Request processed in 53us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.311132:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1162 00000100:00100000:1.0:1769074769.311214:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625122944:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074769.320520:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616803584:12345-192.168.204.60@tcp:4:cat.0 Request processed in 69869us (70233us total) trans 21474841187 rc 0/0 00000100:00100000:1.0:1769074769.320539:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 743 00002000:00100000:1.0:1769074769.320557:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3481600->4194303) req@ffff89b54c847100 x1855007616803584/t21474841187(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/448 e 0 to 0 dl 1769074780 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.320574:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c847100 with x1855007616803584 ext(3481600->4194303) 00010000:00010000:1.0:1769074769.320584:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 25 pid: 28955 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.320605:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 25 pid: 28955 timeout: 1857 lvb_type: 0 00002000:00010000:1.0:1769074769.320614:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54c847100 00000100:00100000:3.0:1769074769.329128:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.329872:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804224 00000100:00100000:0.0:1769074769.329885:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804224 00010000:00010000:0.0:1769074769.329929:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 25 pid: 28955 timeout: 1857 lvb_type: 0 00000100:00100000:0.0:1769074769.329981:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6971 00000100:00100000:0.0:1769074769.329991:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616804224:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074769.330008:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074769.330029:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 25 pid: 28955 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074769.330042:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 25 pid: 28955 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074769.330068:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70400/0x7fe2323adfd53617 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [3481600->18446744073709551615] (req 3481600->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b8f0 expref: 25 pid: 28955 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074769.330087:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074769.330089:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074769.330100:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.330110:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.330119:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074769.330121:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074769.330124:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.330132:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.330178:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074769.330287:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625123072:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074769.330408:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074769.330417:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625123200:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074769.330640:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.342762:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625123072:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074769.342781:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625123200:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074769.342850:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074769.342929:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616804224:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12938us (13813us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074769.342943:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6971 00000100:00100000:2.0:1769074769.357062:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.357236:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804352 00000100:00100000:1.0:1769074769.357250:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804352 00002000:00100000:1.0:1769074769.357311:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67a0f4000 x1855007616804352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.357323:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f4000 with x1855007616804352 ext(3145728->4194303) 00002000:00010000:1.0:1769074769.357330:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f4000 00000100:00100000:1.0:1769074769.357408:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8480 00000100:00100000:1.0:1769074769.357420:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616804352:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074769.373639:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.377516:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616804352:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20086us (20456us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074769.377530:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8480 00002000:00100000:1.0:1769074769.377545:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67a0f4000 x1855007616804352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.377555:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f4000 with x1855007616804352 ext(3145728->4194303) 00002000:00010000:1.0:1769074769.377561:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f4000 00000100:00100000:1.0:1769074769.377587:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804480 00000100:00100000:1.0:1769074769.377594:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804480 00002000:00100000:1.0:1769074769.377626:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54cfefb80 x1855007616804480/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.377634:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfefb80 with x1855007616804480 ext(3145728->4194303) 00002000:00010000:1.0:1769074769.377637:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfefb80 00000100:00100000:0.0:1769074769.381084:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8481 00000100:00100000:0.0:1769074769.381113:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616804480:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074769.394947:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.395927:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804736 00000100:00100000:1.0:1769074769.395939:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804736 00000100:00100000:1.0:1769074769.396009:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13982 00000100:00100000:1.0:1769074769.396019:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616804736:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.396036:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.396054:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd5372f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9bb expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.396119:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123328:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074769.396386:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648c780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.396528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.396719:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804608 00000100:00100000:1.0:1769074769.396725:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804608 00000100:00100000:1.0:1769074769.396764:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13983 00000100:00100000:1.0:1769074769.396772:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616804608:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.396782:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.396797:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9b4 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.396851:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.396853:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.396865:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 29 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.396884:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 29 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.396900:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877400, rc 0) 00000100:00100000:1.0:1769074769.396943:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616804608:12345-192.168.204.60@tcp:101:cat.0 Request processed in 172us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.396952:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13983 00002000:00010000:3.0:1769074769.407313:0:10284:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074765 -> 1769074768 00002000:00010000:3.0:1769074769.407325:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074765 -> 1769074768 00002000:00010000:3.0:1769074769.407328:0:10284:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074765 -> 1769074768 00000100:00100000:3.0:1769074769.407360:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123328:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074769.407434:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd5372f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9bb expref: 25 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.407464:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd5372f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9bb expref: 25 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.407478:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:3.0:1769074769.407937:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616804736:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11915us (12997us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074769.407949:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13982 00000100:00100000:1.0:1769074769.435883:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616804480:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 54769us (62243us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074769.435899:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8481 00002000:00100000:1.0:1769074769.435914:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54cfefb80 x1855007616804480/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.435925:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfefb80 with x1855007616804480 ext(3145728->4194303) 00002000:00010000:1.0:1769074769.435931:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfefb80 00000100:00100000:2.0:1769074769.436335:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.436622:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804864 00000100:00100000:1.0:1769074769.436631:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804864 00002000:00100000:1.0:1769074769.436672:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67a0f6d80 x1855007616804864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.436682:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6d80 with x1855007616804864 ext(4194304->4808703) 00002000:00010000:1.0:1769074769.436687:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f6d80 00000100:00100000:1.0:1769074769.436738:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8482 00000100:00100000:1.0:1769074769.436748:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2407:x1855007616804864:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074769.450145:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2407:x1855007616804864:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13386us (13800us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074769.450161:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8482 00002000:00100000:1.0:1769074769.450174:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67a0f6d80 x1855007616804864/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.450192:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6d80 with x1855007616804864 ext(4194304->4808703) 00002000:00010000:1.0:1769074769.450199:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f6d80 00000100:00100000:0.0:1769074769.455121:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.455277:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616804992 00000100:00100000:3.0:1769074769.455291:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616804992 00000100:00100000:3.0:1769074769.455353:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13984 00000100:00100000:3.0:1769074769.455373:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616804992:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.455394:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.455412:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd5373d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9c2 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.455482:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123456:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074769.455918:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.467802:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:3.0:1769074769.468593:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074768 -> 1769074769 00000100:00100000:3.0:1769074769.468617:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123456:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074769.468653:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd5373d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9c2 expref: 21 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.468675:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd5373d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9c2 expref: 21 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.468687:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9a00, rc 0) 00000100:00100000:3.0:1769074769.468749:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616804992:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13387us (13639us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074769.468758:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13984 00000100:00100000:3.0:1769074769.468786:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805120 00000100:00100000:3.0:1769074769.468793:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805120 00000100:00100000:3.0:1769074769.468881:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13985 00000100:00100000:3.0:1769074769.468892:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616805120:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.468909:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.468930:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53744 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9c9 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.469018:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123584:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074769.469107:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00002000:00010000:3.0:1769074769.474999:0:10284:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074768 -> 1769074769 00002000:00010000:3.0:1769074769.475008:0:10284:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074768 -> 1769074769 00002000:00010000:3.0:1769074769.475011:0:10284:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074768 -> 1769074769 00000100:00100000:3.0:1769074769.475034:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123584:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074769.475093:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53744 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9c9 expref: 25 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.475117:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53744 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9c9 expref: 25 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.475129:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:3.0:1769074769.475201:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616805120:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6311us (7523us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074769.475211:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13985 00000100:00100000:0.0:1769074769.476612:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.476701:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805248 00000100:00100000:1.0:1769074769.476721:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805248 00002000:00100000:1.0:1769074769.476776:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54759f480 x1855007616805248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.476787:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f480 with x1855007616805248 ext(4194304->4808703) 00002000:00010000:1.0:1769074769.476793:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759f480 00000100:00100000:1.0:1769074769.476860:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8483 00000100:00100000:1.0:1769074769.476870:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616805248:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074769.485752:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.485904:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805376 00000100:00100000:1.0:1769074769.485918:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805376 00000100:00100000:1.0:1769074769.486023:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13986 00000100:00100000:1.0:1769074769.486037:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616805376:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.486058:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.486122:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867200/0x7fe2323adfd5374b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9d0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.486194:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123712:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074769.496073:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648cb40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074769.513074:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616805248:12345-192.168.204.60@tcp:3:cat.0 Request processed in 36191us (36455us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074769.513100:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8483 00002000:00100000:1.0:1769074769.513111:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54759f480 x1855007616805248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.513121:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759f480 with x1855007616805248 ext(4194304->4808703) 00002000:00010000:1.0:1769074769.513128:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759f480 00000100:00100000:1.0:1769074769.521139:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123712:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074769.521226:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867200/0x7fe2323adfd5374b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9d0 expref: 25 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.521259:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867200/0x7fe2323adfd5374b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9d0 expref: 25 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.521276:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867200, rc 0) 00000100:00100000:1.0:1769074769.521451:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616805376:12345-192.168.204.60@tcp:101:cat.0 Request processed in 35417us (35711us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074769.521462:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13986 00000100:00100000:1.0:1769074769.534223:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.535566:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805504 00000100:00100000:3.0:1769074769.535584:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805504 00000100:00100000:3.0:1769074769.535687:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13987 00000100:00100000:3.0:1769074769.535702:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b677578450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616805504:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.536296:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.536328:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd53752 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9d7 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.536425:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123840:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074769.536533:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00002000:00010000:3.0:1769074769.545568:0:10284:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074768 -> 1769074769 00002000:00010000:3.0:1769074769.545581:0:10284:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074768 -> 1769074769 00000100:00100000:3.0:1769074769.545628:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfee300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123840:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074769.545680:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd53752 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9d7 expref: 21 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.548189:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd53752 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9d7 expref: 21 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.548220:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9200, rc 0) 00000100:00100000:2.0:1769074769.548739:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b677578450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616805504:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13031us (14520us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074769.548749:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13987 00000100:00100000:0.0:1769074769.559141:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.559244:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805632 00000100:00100000:2.0:1769074769.559256:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805632 00000100:00100000:2.0:1769074769.559337:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13988 00000100:00100000:2.0:1769074769.559349:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616805632:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074769.559381:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074769.559405:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53759 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9de expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.559484:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123968:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074769.559600:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.564098:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625123968:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074769.564144:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53759 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9de expref: 21 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074769.564168:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53759 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9de expref: 21 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074769.564272:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:2.0:1769074769.564356:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616805632:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5009us (5224us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074769.564377:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13988 00000100:00100000:1.0:1769074769.573005:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.573379:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805760 00000100:00100000:2.0:1769074769.573391:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805760 00000100:00100000:2.0:1769074769.573470:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13989 00000100:00100000:2.0:1769074769.573482:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62788:x1855007616805760:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074769.573507:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074769.573535:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9e5 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.573574:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.573587:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 30 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.573595:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.573598:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.573602:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3039232->3145727] (req 3039232->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.573640:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 30 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.573655:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 30 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.573670:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625124096:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.573794:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.573804:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 16 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.573863:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625124224:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074769.573895:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.585758:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.586060:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616805888 00000100:00100000:3.0:1769074769.586073:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616805888 00010000:00010000:3.0:1769074769.586118:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 31 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:3.0:1769074769.586157:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6972 00000100:00100000:3.0:1769074769.586166:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616805888:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074769.586203:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074769.586231:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 31 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.586248:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 31 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.586276:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074769.586371:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616805888:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 193us (610us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.586381:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6972 00010000:00010000:2.0:1769074769.589684:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd536f7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b998 expref: 30 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.589723:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625124096:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074769.589739:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625124224:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.589770:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3039232->3145727] (req 3039232->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 29 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.589789:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:2.0:1769074769.589935:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62788:x1855007616805760:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 16454us (16944us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.589944:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13989 00000100:00100000:0.0:1769074769.609096:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.609248:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806016 00000100:00100000:1.0:1769074769.609266:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806016 00010000:00010000:1.0:1769074769.609321:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 17 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:1.0:1769074769.609387:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6973 00000100:00100000:1.0:1769074769.609401:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616806016:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074769.609423:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074769.609453:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 17 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.609469:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 17 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.609498:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd536db lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b97c expref: 17 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.609515:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.609518:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074769.609527:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.609538:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.609583:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 28 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.609599:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625124352:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074769.609701:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64648cb40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074769.613481:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625124352:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074769.613516:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.613577:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616806016:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4179us (4492us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.613588:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6973 00000100:00100000:0.0:1769074769.616777:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.616919:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806144 00000100:00100000:1.0:1769074769.616930:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806144 00002000:00100000:1.0:1769074769.616989:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3039232->3043327) req@ffff89b54759ca80 x1855007616806144/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.616999:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759ca80 with x1855007616806144 ext(3039232->3043327) 00002000:00010000:1.0:1769074769.617005:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759ca80 00000100:00100000:1.0:1769074769.617028:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8484 00000100:00100000:1.0:1769074769.617037:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616806144:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074769.617210:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616806144:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 173us (445us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074769.617218:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8484 00002000:00100000:1.0:1769074769.617225:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3039232->3043327) req@ffff89b54759ca80 x1855007616806144/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/4536 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.617233:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759ca80 with x1855007616806144 ext(3039232->3043327) 00002000:00010000:1.0:1769074769.617236:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759ca80 00000100:00100000:0.0:1769074769.626689:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.626793:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806272 00000100:00100000:2.0:1769074769.626837:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806272 00000100:00100000:2.0:1769074769.626890:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13990 00000100:00100000:2.0:1769074769.626901:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62788:x1855007616806272:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074769.626920:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074769.626940:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9ec expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.626970:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.626979:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.626985:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.626993:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 19 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.626999:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:2.0:1769074769.627001:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:2.0:1769074769.627005:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 7/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->2994175] (req 2097152->2994175) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 25 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.627038:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 19 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.627052:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 19 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.627066:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124480:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.627166:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.627186:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 25 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.627196:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124608:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.627229:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.627242:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 25 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.627251:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124736:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.627279:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.627287:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.627295:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124864:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074769.627315:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.634138:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.634213:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.634444:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124480:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074769.634461:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124608:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074769.635355:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806528 00000100:00100000:1.0:1769074769.635376:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806528 00010000:00010000:1.0:1769074769.635451:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 26 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:1.0:1769074769.635507:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806400 00000100:00100000:1.0:1769074769.635511:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806400 00010000:00010000:1.0:1769074769.635529:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 20 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:1.0:1769074769.635575:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6974 00000100:00100000:1.0:1769074769.635585:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616806528:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074769.635607:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074769.635638:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 26 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.635651:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 26 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.635682:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd536e2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b983 expref: 26 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.635702:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.635773:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616806528:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 189us (1648us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.635781:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6974 00000100:00100000:1.0:1769074769.635854:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6975 00000100:00100000:1.0:1769074769.635865:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616806400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074769.635875:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074769.635885:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 20 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.635896:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 20 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.635915:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd536f0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b991 expref: 20 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:1.0:1769074769.635926:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.635948:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616806400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 86us (1735us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.635955:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6975 00000100:00100000:2.0:1769074769.640125:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124736:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074769.640142:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625124864:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.640191:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->2994175] (req 2097152->2994175) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 24 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.640215:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47e00, rc 0) 00000100:00100000:2.0:1769074769.640283:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62788:x1855007616806272:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 13384us (13604us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.640296:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13990 00000100:00100000:0.0:1769074769.643754:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.643922:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806656 00000100:00100000:2.0:1769074769.643933:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806656 00000100:00100000:2.0:1769074769.644002:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13991 00000100:00100000:2.0:1769074769.644012:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62795:x1855007616806656:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074769.644029:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074769.644048:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9f3 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.644078:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 28 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.644087:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.644095:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.644097:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.644101:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1220608->1720319] (req 1220608->1720319) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 17 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.644134:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.644146:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 17 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.644161:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625124992:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.644984:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 28 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.644998:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 28 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.645009:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625125120:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074769.645048:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.650596:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.650857:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806784 00000100:00100000:2.0:1769074769.650870:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806784 00010000:00010000:2.0:1769074769.650913:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.650960:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6976 00000100:00100000:2.0:1769074769.650981:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616806784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074769.650998:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074769.651020:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.651033:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.651055:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53728 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ad expref: 24 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.651072:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074769.651138:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616806784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 168us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.651146:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6976 00000100:00100000:2.0:1769074769.653307:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625124992:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074769.667510:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.667543:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.667580:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.667610:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616806912 00000100:00100000:3.0:1769074769.667619:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616806912 00010000:00010000:3.0:1769074769.667661:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 1857 lvb_type: 0 00000100:00100000:3.0:1769074769.667695:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807040 00000100:00100000:3.0:1769074769.667699:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807040 00010000:00010000:3.0:1769074769.667720:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 18 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:3.0:1769074769.667739:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807168 00000100:00100000:3.0:1769074769.667742:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807168 00010000:00010000:3.0:1769074769.667755:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 29 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:3.0:1769074769.667772:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6977 00000100:00100000:3.0:1769074769.667781:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616806912:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074769.667795:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074769.667860:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.667875:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.667900:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd536fe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b99f expref: 19 pid: 9048 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.667918:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074769.667920:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074769.667926:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 22 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.667936:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 22 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.667975:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 22 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.667990:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfedc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625125248:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074769.668064:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.668085:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6978 00000100:00100000:3.0:1769074769.668092:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616807040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074769.668102:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074769.668115:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 18 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.668127:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 18 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.668139:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd536bf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b960 expref: 18 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.668153:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074769.668178:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616807040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 87us (634us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.668186:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6978 00000100:00100000:3.0:1769074769.668205:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6979 00000100:00100000:3.0:1769074769.668211:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616807168:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074769.668218:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074769.668225:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 29 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.668235:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 29 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.668246:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074769.668247:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074769.668252:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.668259:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.668280:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.668289:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625125376:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074769.668314:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00010000:00010000:2.0:1769074769.668719:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd5368e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b94b expref: 29 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.668731:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625125120:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074769.668758:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 16 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.668768:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47400, rc 0) 00000100:00100000:2.0:1769074769.668839:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62795:x1855007616806656:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 24796us (25067us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.668847:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13991 00000100:00100000:1.0:1769074769.674198:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625125376:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074769.674227:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074769.674292:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616807168:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6078us (6709us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.674304:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6979 00000100:00100000:2.0:1769074769.676768:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfedc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625125248:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074769.676844:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074769.676896:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616806912:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9115us (9394us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.676907:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6977 00000100:00100000:2.0:1769074769.691298:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.691410:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807296 00000100:00100000:0.0:1769074769.691421:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807296 00002000:00100000:0.0:1769074769.691476:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1220608->1224703) req@ffff89b67a0f6300 x1855007616807296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.691488:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6300 with x1855007616807296 ext(1220608->1224703) 00002000:00010000:0.0:1769074769.691494:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f6300 00000100:00100000:0.0:1769074769.691533:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8485 00000100:00100000:0.0:1769074769.691543:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616807296:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074769.691728:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616807296:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 186us (439us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074769.691736:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8485 00002000:00100000:0.0:1769074769.691743:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1220608->1224703) req@ffff89b67a0f6300 x1855007616807296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/4536 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.691751:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f6300 with x1855007616807296 ext(1220608->1224703) 00002000:00010000:0.0:1769074769.691754:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f6300 00000100:00100000:3.0:1769074769.718942:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.719057:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807552 00000100:00100000:1.0:1769074769.719066:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807552 00000100:00100000:1.0:1769074769.719123:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13310 00000100:00100000:1.0:1769074769.719133:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62800:x1855007616807552:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.719165:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.719203:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53775 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879b9fa expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.719285:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719301:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719312:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719328:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719334:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719339:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719487:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd5378a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719498:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd5378a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719505:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd5378a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074769.719515:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074769.719614:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719621:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719626:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719632:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719637:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074769.719641:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719648:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719660:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074769.719662:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074769.719666:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719672:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5377c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719684:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719688:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719693:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719697:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074769.719699:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719704:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719710:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074769.719712:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074769.719714:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719720:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53783 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074769.719764:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd53775 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9fa expref: 234 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.719780:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd5378a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9fa expref: 233 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.719788:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074769.719790:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074769.719793:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866000, rc 0) 00000100:00100000:1.0:1769074769.721272:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62800:x1855007616807552:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2127us (2331us total) trans 8589943927 rc 0/0 00000100:00100000:1.0:1769074769.721287:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13310 00000100:00100000:1.0:1769074769.733229:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.733351:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807680 00000100:00100000:0.0:1769074769.733371:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807680 00000100:00100000:1.0:1769074769.733378:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00100000:0.0:1769074769.733422:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2990080->2994175) req@ffff89b54441f480 x1855007616807680/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.733433:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f480 with x1855007616807680 ext(2990080->2994175) 00002000:00010000:0.0:1769074769.733438:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441f480 00000100:00100000:1.0:1769074769.735151:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.735193:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.735230:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.735279:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807936 00000100:00100000:1.0:1769074769.735290:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807936 00000100:00100000:1.0:1769074769.735341:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808192 00000100:00100000:1.0:1769074769.735344:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808192 00000100:00100000:1.0:1769074769.735377:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808064 00000100:00100000:1.0:1769074769.735381:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808064 00000100:00100000:1.0:1769074769.735404:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13992 00000100:00100000:1.0:1769074769.735414:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616807936:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:3.0:1769074769.735649:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13993 00000100:00100000:3.0:1769074769.735658:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616808192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.735667:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.735687:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53798 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba0f expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.735839:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.735851:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd5379f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba01 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.735913:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625125504:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074769.735996:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074769.736032:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13994 00000100:00100000:1.0:1769074769.736039:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616808064:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.736049:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.736060:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd537a6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba08 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.736088:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625125632:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074769.736116:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.736383:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625125760:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074769.736414:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.737271:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616807808 00000100:00100000:2.0:1769074769.737281:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616807808 00002000:00100000:2.0:1769074769.737323:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1716224->1720319) req@ffff89b54441d880 x1855007616807808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.737338:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441d880 with x1855007616807808 ext(1716224->1720319) 00002000:00010000:2.0:1769074769.737343:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441d880 00000100:00100000:2.0:1769074769.737409:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8486 00000100:00100000:2.0:1769074769.737418:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616807680:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074769.737586:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616807680:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 169us (4367us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074769.737595:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8486 00002000:00100000:2.0:1769074769.737601:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2990080->2994175) req@ffff89b54441f480 x1855007616807680/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/4536 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.737609:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441f480 with x1855007616807680 ext(2990080->2994175) 00002000:00010000:2.0:1769074769.737612:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441f480 00000100:00100000:2.0:1769074769.737630:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8487 00000100:00100000:2.0:1769074769.737637:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616807808:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074769.737701:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616807808:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 64us (4322us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074769.737897:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8487 00002000:00100000:2.0:1769074769.737904:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1716224->1720319) req@ffff89b54441d880 x1855007616807808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/4536 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.737912:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441d880 with x1855007616807808 ext(1716224->1720319) 00002000:00010000:2.0:1769074769.737916:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441d880 00002000:00010000:2.0:1769074769.746733:0:28954:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074768 -> 1769074769 00002000:00010000:2.0:1769074769.746744:0:28954:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074768 -> 1769074769 00002000:00010000:2.0:1769074769.746746:0:28954:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074768 -> 1769074769 00000100:00100000:1.0:1769074769.746759:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625125504:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074769.746759:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625125632:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074769.746799:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd5379f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba01 expref: 21 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074769.746799:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd537a6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba08 expref: 17 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.746844:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd5379f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba01 expref: 21 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074769.746844:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd537a6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba08 expref: 17 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.746859:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00010000:00010000:2.0:1769074769.746859:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867200, rc 0) 00000100:00100000:1.0:1769074769.746922:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616807936:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11508us (11779us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074769.746922:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616808064:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10881us (11689us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074769.746932:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13994 00000100:00100000:1.0:1769074769.746936:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13992 00002000:00010000:0.0:1769074769.750483:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074768 -> 1769074769 00002000:00010000:0.0:1769074769.750493:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074768 -> 1769074769 00002000:00010000:0.0:1769074769.750495:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074768 -> 1769074769 00000100:00100000:0.0:1769074769.750520:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625125760:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074769.750566:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53798 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba0f expref: 19 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.750591:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53798 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba0f expref: 19 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.750605:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:0.0:1769074769.750686:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616808192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15029us (15491us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074769.750695:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13993 00000100:00100000:3.0:1769074769.761014:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.761130:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808576 00000100:00100000:0.0:1769074769.761142:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808576 00000100:00100000:0.0:1769074769.761210:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4658 00000100:00100000:0.0:1769074769.761221:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62800:x1855007616808576:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:0.0:1769074769.761514:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62800:x1855007616808576:12345-192.168.204.60@tcp:35:cat.0 Request processed in 294us (513us total) trans 8589943928 rc 0/0 00000100:00100000:0.0:1769074769.761522:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4658 00000100:00100000:3.0:1769074769.766566:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.766679:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808320 00000100:00100000:2.0:1769074769.766690:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808320 00000100:00100000:2.0:1769074769.766767:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1000 00000100:00100000:2.0:1769074769.766777:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:2409:x1855007616808320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074769.766794:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074769.766853:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd5378a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9fa expref: 234 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.766875:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074769.766879:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074769.766942:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:2409:x1855007616808320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 166us (387us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.766950:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1000 00000100:00100000:3.0:1769074769.779640:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.779709:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808832 00000100:00100000:1.0:1769074769.779720:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808832 00002000:00100000:1.0:1769074769.779780:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cfec380 x1855007616808832/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 504/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074769.779792:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfec380 with x1855007616808832 ext(1048576->2097151) 00002000:00010000:1.0:1769074769.779798:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfec380 00000100:00100000:1.0:1769074769.779876:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8488 00000100:00100000:1.0:1769074769.779886:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616808832:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074769.793315:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.793879:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808960 00000100:00100000:0.0:1769074769.793895:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808960 00000100:00100000:0.0:1769074769.793944:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13995 00000100:00100000:0.0:1769074769.793954:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616808960:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074769.793981:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074769.794001:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd537ad lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba1d expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074769.794064:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625125888:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074769.794171:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.795685:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.795835:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616808704 00000100:00100000:1.0:1769074769.795845:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616808704 00000100:00100000:1.0:1769074769.795897:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13996 00000100:00100000:1.0:1769074769.795905:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616808704:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.795925:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.795948:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537b4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba16 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074769.796020:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625126016:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074769.797020:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.798110:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616808832:12345-192.168.204.60@tcp:3:cat.0 Request processed in 18217us (18471us total) trans 0 rc 577536/577536 00000100:00100000:2.0:1769074769.798124:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8488 00002000:00100000:2.0:1769074769.798135:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cfec380 x1855007616808832/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 504/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 577536/577536 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.798145:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfec380 with x1855007616808832 ext(1048576->2097151) 00002000:00010000:2.0:1769074769.798151:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfec380 00002000:00010000:0.0:1769074769.802668:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074768 -> 1769074769 00002000:00010000:0.0:1769074769.802676:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074768 -> 1769074769 00002000:00010000:0.0:1769074769.802679:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074768 -> 1769074769 00000100:00100000:0.0:1769074769.802700:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625125888:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074769.802746:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd537ad lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba1d expref: 29 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.802767:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b437400/0x7fe2323adfd537ad lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba1d expref: 29 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074769.802785:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437400, rc 0) 00000100:00100000:0.0:1769074769.802885:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616808960:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8933us (9585us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074769.802895:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13995 00000100:00100000:1.0:1769074769.806143:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625126016:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074769.806189:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537b4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba16 expref: 24 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.806214:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537b4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba16 expref: 24 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074769.806228:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:1.0:1769074769.806327:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616808704:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10420us (10646us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074769.806341:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13996 00000100:00100000:3.0:1769074769.819017:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.819127:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809088 00000100:00100000:1.0:1769074769.819139:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809088 00000100:00100000:1.0:1769074769.819227:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13997 00000100:00100000:1.0:1769074769.819238:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62802:x1855007616809088:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074769.819262:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074769.819282:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba24 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.819314:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.819322:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074769.819324:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074769.819328:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.819373:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.819387:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 22 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:1.0:1769074769.819402:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625126144:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074769.819511:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.823465:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074769.823489:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625126272:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074769.823557:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074769.823860:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625126144:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074769.823893:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074769.823914:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:1.0:1769074769.823971:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62802:x1855007616809088:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4735us (4964us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074769.823981:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13997 00000100:00100000:3.0:1769074769.824851:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625126272 00000100:00100000:3.0:1769074769.824863:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625126272 00000100:00100000:3.0:1769074769.824915:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1163 00000100:00100000:3.0:1769074769.824925:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625126272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074769.824953:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1060980032 pending 2543616 free 3900067840 avail 3668545536 00000100:00100000:3.0:1769074769.825016:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625126272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 93us (1458us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.825022:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1163 00000100:00100000:3.0:1769074769.825069:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625126272:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074769.825887:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.826510:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809216 00000100:00100000:3.0:1769074769.826519:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809216 00002000:00100000:3.0:1769074769.826567:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1380352->1490943) req@ffff89b67a0f4a80 x1855007616809216/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074769.826578:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f4a80 with x1855007616809216 ext(1380352->1490943) 00010000:00010000:3.0:1769074769.826587:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 23 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074769.826602:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 23 pid: 8396 timeout: 1857 lvb_type: 0 00002000:00010000:3.0:1769074769.826609:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a0f4a80 00000100:00100000:3.0:1769074769.826646:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 744 00000100:00100000:3.0:1769074769.826655:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616809216:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074769.829961:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.830613:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809344 00000100:00100000:3.0:1769074769.830627:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809344 00002000:00100000:3.0:1769074769.830689:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54759e680 x1855007616809344/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074769.830862:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759e680 with x1855007616809344 ext(0->1048575) 00002000:00010000:3.0:1769074769.830870:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759e680 00000100:00100000:3.0:1769074769.830976:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8489 00000100:00100000:3.0:1769074769.830987:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616809344:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074769.842615:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616809216:12345-192.168.204.60@tcp:4:cat.0 Request processed in 15948us (16720us total) trans 21474841188 rc 0/0 00000100:00100000:0.0:1769074769.842631:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 744 00002000:00100000:0.0:1769074769.842645:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1380352->1490943) req@ffff89b67a0f4a80 x1855007616809216/t21474841188(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/448 e 0 to 0 dl 1769074780 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.842656:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f4a80 with x1855007616809216 ext(1380352->1490943) 00010000:00010000:0.0:1769074769.842666:0:16436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 25 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074769.842683:0:16436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 25 pid: 8396 timeout: 1857 lvb_type: 0 00002000:00010000:0.0:1769074769.842691:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a0f4a80 00000100:00100000:3.0:1769074769.848272:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.848860:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809472 00000100:00100000:2.0:1769074769.848871:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809472 00010000:00010000:2.0:1769074769.848915:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 25 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.848962:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6980 00000100:00100000:2.0:1769074769.848971:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616809472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074769.848987:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074769.849010:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 25 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.849023:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 25 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.849049:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46600/0x7fe2323adfd536b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1490943) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b959 expref: 25 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.849071:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.849073:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.849080:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.849089:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.849134:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 18 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.849147:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625126400:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074769.849385:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.849786:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616809344:12345-192.168.204.60@tcp:3:cat.0 Request processed in 18799us (19838us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074769.849797:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8489 00002000:00100000:0.0:1769074769.849837:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b54759e680 x1855007616809344/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.849852:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759e680 with x1855007616809344 ext(0->1048575) 00002000:00010000:0.0:1769074769.849857:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759e680 00000100:00100000:3.0:1769074769.872594:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625126400:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074769.872627:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074769.872931:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.873021:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.873053:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809600 00000100:00100000:0.0:1769074769.873061:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809600 00002000:00100000:0.0:1769074769.873114:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54759d500 x1855007616809600/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.873127:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759d500 with x1855007616809600 ext(1048576->2097151) 00002000:00010000:0.0:1769074769.873133:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759d500 00000100:00100000:0.0:1769074769.873156:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809728 00000100:00100000:0.0:1769074769.873159:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809728 00002000:00100000:0.0:1769074769.873189:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54759a300 x1855007616809728/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 504/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.873197:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759a300 with x1855007616809728 ext(1048576->2097151) 00002000:00010000:0.0:1769074769.873201:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759a300 00000100:00100000:0.0:1769074769.873225:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8490 00000100:00100000:0.0:1769074769.873234:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616809600:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074769.873896:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8491 00000100:00100000:0.0:1769074769.873906:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616809728:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074769.875771:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616809472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 23729us (24439us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.875828:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6980 00000100:00100000:2.0:1769074769.888208:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616809600:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14962us (15264us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074769.888223:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8490 00002000:00100000:2.0:1769074769.888238:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54759d500 x1855007616809600/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.888250:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759d500 with x1855007616809600 ext(1048576->2097151) 00002000:00010000:2.0:1769074769.888255:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54759d500 00000100:00100000:1.0:1769074769.893508:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.893622:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809856 00000100:00100000:0.0:1769074769.893634:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809856 00000100:00100000:0.0:1769074769.893699:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13998 00000100:00100000:0.0:1769074769.893721:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62800:x1855007616809856:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074769.893742:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074769.893762:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba2b expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.893794:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 18 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.893803:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074769.893863:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074769.893869:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.893904:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 18 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.893918:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 18 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:0.0:1769074769.893931:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625126528:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074769.894011:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.896197:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616809728:12345-192.168.204.60@tcp:3:cat.0 Request processed in 22283us (23165us total) trans 0 rc 548864/548864 00000100:00100000:2.0:1769074769.896210:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8491 00002000:00100000:2.0:1769074769.896222:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54759a300 x1855007616809728/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 504/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 548864/548864 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.896231:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759a300 with x1855007616809728 ext(1048576->2097151) 00002000:00010000:2.0:1769074769.896238:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759a300 00000100:00100000:1.0:1769074769.903304:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074769.903431:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625126528:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074769.903471:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074769.903489:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437e00, rc 0) 00000100:00100000:0.0:1769074769.903562:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62800:x1855007616809856:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9843us (10066us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074769.903572:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13998 00000100:00100000:0.0:1769074769.904048:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616809984 00000100:00100000:0.0:1769074769.904055:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616809984 00000100:00100000:3.0:1769074769.906910:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13999 00000100:00100000:1.0:1769074769.907345:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.907472:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.907518:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810112 00000100:00100000:3.0:1769074769.907528:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810112 00000100:00100000:3.0:1769074769.907571:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14000 00000100:00100000:3.0:1769074769.907578:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616810112:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.907595:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.907612:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537c9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba39 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074769.907669:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625126656:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074769.907749:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.910469:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616809984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074769.910516:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074769.910538:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd537d0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba32 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.910598:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625126784:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074769.910683:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074769.913406:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810240 00000100:00100000:2.0:1769074769.913422:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810240 00002000:00100000:2.0:1769074769.913483:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1134592->1605631) req@ffff89b5497a9500 x1855007616810240/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.913494:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9500 with x1855007616810240 ext(1134592->1605631) 00010000:00010000:2.0:1769074769.913502:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.913518:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00002000:00010000:2.0:1769074769.913525:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5497a9500 00000100:00100000:2.0:1769074769.913562:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 745 00000100:00100000:2.0:1769074769.913571:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616810240:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074769.914721:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625126656:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074769.914760:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537c9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba39 expref: 22 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.914782:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537c9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba39 expref: 22 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074769.914794:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8a00, rc 0) 00000100:00100000:2.0:1769074769.915676:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625126784:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074769.915712:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd537d0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba32 expref: 17 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074769.915729:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd537d0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba32 expref: 17 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074769.915740:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46600, rc 0) 00000100:00100000:2.0:1769074769.915771:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616809984:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8847us (12478us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074769.915779:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13999 00000100:00100000:3.0:1769074769.920940:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616810112:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13346us (13580us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074769.920958:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14000 00000100:00100000:2.0:1769074769.929054:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616810240:12345-192.168.204.60@tcp:4:cat.0 Request processed in 15472us (21569us total) trans 21474841189 rc 0/0 00000100:00100000:2.0:1769074769.929069:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 745 00002000:00100000:2.0:1769074769.929085:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1134592->1605631) req@ffff89b5497a9500 x1855007616810240/t21474841189(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/448 e 0 to 0 dl 1769074780 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.929097:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a9500 with x1855007616810240 ext(1134592->1605631) 00010000:00010000:2.0:1769074769.929108:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.929126:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00002000:00010000:2.0:1769074769.929133:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5497a9500 00000100:00100000:1.0:1769074769.933475:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074769.933572:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810368 00000100:00100000:3.0:1769074769.933593:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810368 00000100:00100000:3.0:1769074769.933656:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14001 00000100:00100000:3.0:1769074769.933664:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62802:x1855007616810368:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074769.933682:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074769.933711:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba40 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.933752:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 27 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.933760:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074769.933762:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074769.933765:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.933800:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 27 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.933854:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 27 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:3.0:1769074769.933867:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625126912:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074769.933952:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074769.937416:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625126912:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074769.937483:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074769.937506:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8a00, rc 0) 00000100:00100000:3.0:1769074769.937596:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62802:x1855007616810368:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3932us (4131us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.937606:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14001 00000100:00100000:1.0:1769074769.942650:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.942857:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810496 00000100:00100000:2.0:1769074769.942871:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810496 00002000:00100000:2.0:1769074769.942935:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3039232->3145727) req@ffff89b549f29c00 x1855007616810496/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.942948:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f29c00 with x1855007616810496 ext(3039232->3145727) 00010000:00010000:2.0:1769074769.942957:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 28 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.942971:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 28 pid: 10284 timeout: 1857 lvb_type: 0 00002000:00010000:2.0:1769074769.942979:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f29c00 00000100:00100000:2.0:1769074769.943005:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 746 00000100:00100000:2.0:1769074769.943014:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616810496:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074769.946277:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.947235:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810624 00000100:00100000:2.0:1769074769.947246:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810624 00010000:00010000:2.0:1769074769.947285:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.947329:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6981 00000100:00100000:2.0:1769074769.947338:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616810624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074769.947362:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074769.947384:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.947397:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.947420:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd536e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1134592->1605631) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b98a expref: 21 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.947437:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.947451:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.947461:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.947471:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.947512:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.947527:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625127040:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074769.947619:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4529c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.949597:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616810496:12345-192.168.204.60@tcp:4:cat.0 Request processed in 6570us (6947us total) trans 17179874502 rc 0/0 00000100:00100000:0.0:1769074769.949612:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 746 00002000:00100000:0.0:1769074769.949626:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3039232->3145727) req@ffff89b549f29c00 x1855007616810496/t17179874502(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/448 e 0 to 0 dl 1769074780 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.949635:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f29c00 with x1855007616810496 ext(3039232->3145727) 00010000:00010000:0.0:1769074769.949646:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074769.949664:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 1857 lvb_type: 0 00002000:00010000:0.0:1769074769.949671:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b549f29c00 00000100:00100000:1.0:1769074769.957071:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074769.957212:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.957362:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625127040:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074769.957395:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074769.957491:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616810624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10153us (11224us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074769.957502:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6981 00000100:00100000:2.0:1769074769.957564:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810752 00000100:00100000:2.0:1769074769.957571:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810752 00010000:00010000:2.0:1769074769.957602:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074769.957629:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6982 00000100:00100000:2.0:1769074769.957638:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616810752:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074769.957652:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074769.957672:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.957685:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.957720:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53760 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3039232->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9e5 expref: 30 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074769.957736:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.957739:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074769.957747:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.957757:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074769.957794:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074769.957861:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625127168:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074769.957913:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b039c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074769.958716:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616810880 00000100:00100000:0.0:1769074769.958728:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616810880 00002000:00100000:0.0:1769074769.958778:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547ef7100 x1855007616810880/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.958788:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7100 with x1855007616810880 ext(1048576->2097151) 00002000:00010000:0.0:1769074769.958794:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef7100 00000100:00100000:0.0:1769074769.958859:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8492 00000100:00100000:0.0:1769074769.958869:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616810880:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074769.976627:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625127168:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074769.976665:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074769.976744:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616810752:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 19106us (19690us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074769.976756:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6982 00000100:00100000:1.0:1769074769.991429:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074769.991902:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811008 00000100:00100000:2.0:1769074769.991912:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811008 00002000:00100000:2.0:1769074769.991962:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b547ef5180 x1855007616811008/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/0 e 0 to 0 dl 1769074780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074769.991979:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5180 with x1855007616811008 ext(2097152->3145727) 00002000:00010000:2.0:1769074769.991985:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547ef5180 00000100:00100000:2.0:1769074769.992015:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8493 00000100:00100000:2.0:1769074769.992023:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616811008:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074769.993518:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616810880:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34636us (36291us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074769.993536:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8492 00002000:00100000:0.0:1769074769.993550:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b547ef7100 x1855007616810880/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074769.993561:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef7100 with x1855007616810880 ext(1048576->2097151) 00002000:00010000:0.0:1769074769.993568:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef7100 00000100:00100000:1.0:1769074770.024004:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.024190:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616811008:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32165us (32771us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074770.024202:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8493 00002000:00100000:2.0:1769074770.024215:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b547ef5180 x1855007616811008/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:305/0 lens 488/440 e 0 to 0 dl 1769074780 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.024226:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5180 with x1855007616811008 ext(2097152->3145727) 00002000:00010000:2.0:1769074770.024232:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547ef5180 00000100:00100000:3.0:1769074770.024657:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811136 00000100:00100000:3.0:1769074770.024666:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811136 00000100:00100000:3.0:1769074770.024743:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14002 00000100:00100000:3.0:1769074770.024752:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62800:x1855007616811136:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074770.024767:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074770.024783:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba47 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.024832:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.024835:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.024843:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 26 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.024858:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 26 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.024870:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:3.0:1769074770.024901:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62800:x1855007616811136:12345-192.168.204.60@tcp:101:cat.0 Request processed in 151us (908us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.024908:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14002 00000100:00080000:0.0:1769074770.025401:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074770.036143:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.036331:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811264 00000100:00100000:2.0:1769074770.036343:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811264 00000100:00100000:2.0:1769074770.036422:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14003 00000100:00100000:2.0:1769074770.036432:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62802:x1855007616811264:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.036456:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.036477:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba4e expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.036511:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 25 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.036519:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.036521:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.036526:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.036561:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 25 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.036574:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 25 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074770.036589:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625127296:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.036681:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b039c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.056957:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.058923:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811392 00000100:00100000:2.0:1769074770.058940:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811392 00002000:00100000:2.0:1769074770.059004:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5444a5500 x1855007616811392/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.059016:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5500 with x1855007616811392 ext(1048576->2097151) 00002000:00010000:2.0:1769074770.059022:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5500 00000100:00100000:2.0:1769074770.059073:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8494 00000100:00100000:2.0:1769074770.059084:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616811392:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074770.070288:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.070331:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.070395:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811776 00000100:00100000:3.0:1769074770.070406:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811776 00002000:00100000:3.0:1769074770.070467:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2994175) req@ffff89b54cfee300 x1855007616811776/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.070478:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfee300 with x1855007616811776 ext(2097152->2994175) 00010000:00010000:3.0:1769074770.070487:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074770.070504:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1857 lvb_type: 0 00002000:00010000:3.0:1769074770.070511:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cfee300 00000100:00100000:3.0:1769074770.070538:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 747 00000100:00100000:3.0:1769074770.070547:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616811776:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074770.071261:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811648 00000100:00100000:0.0:1769074770.071269:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811648 00000100:00100000:0.0:1769074770.071322:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13311 00000100:00100000:0.0:1769074770.071331:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62807:x1855007616811648:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074770.071365:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074770.071386:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd537ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba55 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.071467:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071482:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071493:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071509:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071516:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071572:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071683:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53801 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071694:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53801 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071714:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53801 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074770.071726:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074770.071845:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071852:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071858:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071865:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071870:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b437600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074770.071874:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071881:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071894:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074770.071898:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074770.071904:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071909:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437600/0x7fe2323adfd537f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071923:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071929:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071935:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071940:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba44600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074770.071942:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071948:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071953:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47700) 00010000:00010000:0.0:1769074770.071956:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074770.071959:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.071964:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd537fa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074770.072010:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437400/0x7fe2323adfd537ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba55 expref: 236 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.072025:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53801 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba55 expref: 235 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.072032:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074770.072035:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074770.072037:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45600, rc 0) 00000100:00100000:0.0:1769074770.072106:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62807:x1855007616811648:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 775us (1830us total) trans 8589943929 rc 0/0 00000100:00100000:0.0:1769074770.072115:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13311 00000100:00080000:3.0:1769074770.072597:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074770.073890:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625127296:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.073932:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.073953:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:2.0:1769074770.074023:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62802:x1855007616811264:12345-192.168.204.60@tcp:101:cat.0 Request processed in 37592us (37896us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.074033:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14003 00000100:00100000:3.0:1769074770.080009:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.080187:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616811904 00000100:00100000:2.0:1769074770.080199:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616811904 00000100:00100000:2.0:1769074770.080261:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14004 00000100:00100000:2.0:1769074770.080270:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62808:x1855007616811904:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074770.080290:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.080308:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba5c expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.080337:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.080346:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.080348:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.080364:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2351104->2510847] (req 2351104->2510847) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.080424:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.080436:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074770.080465:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625127424:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.080546:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074770.095987:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616811392:12345-192.168.204.60@tcp:3:cat.0 Request processed in 36894us (39031us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074770.096003:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8494 00002000:00100000:2.0:1769074770.096015:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5444a5500 x1855007616811392/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.096025:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5500 with x1855007616811392 ext(1048576->2097151) 00002000:00010000:2.0:1769074770.096031:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5500 00000100:00100000:2.0:1769074770.101802:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625127424:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.101902:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2351104->2510847] (req 2351104->2510847) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 18 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.101925:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46600, rc 0) 00000100:00100000:2.0:1769074770.102003:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+18:62808:x1855007616811904:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 21732us (22004us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.102012:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14004 00000100:00100000:1.0:1769074770.104939:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.104986:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.105015:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.105209:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812416 00000100:00100000:2.0:1769074770.105220:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812416 00000100:00100000:2.0:1769074770.105290:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4659 00000100:00100000:2.0:1769074770.105299:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62807:x1855007616812416:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074770.105610:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:62807:x1855007616812416:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 310us (681us total) trans 8589943930 rc 0/0 00000100:00100000:2.0:1769074770.105620:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4659 00000100:00100000:0.0:1769074770.106267:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812032 00000100:00100000:0.0:1769074770.106279:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812032 00010000:00010000:0.0:1769074770.106316:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 1857 lvb_type: 0 00000100:00100000:0.0:1769074770.106341:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812160 00000100:00100000:0.0:1769074770.106343:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812160 00000100:00100000:0.0:1769074770.106390:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6983 00000100:00100000:0.0:1769074770.106399:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616812032:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074770.106415:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074770.106453:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074770.106468:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074770.106495:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8a00/0x7fe2323adfd537d7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba40 expref: 18 pid: 28954 timeout: 1857 lvb_type: 0 00010000:00010000:0.0:1769074770.106511:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074770.106513:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074770.106520:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.106529:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.106566:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074770.106580:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625127552:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074770.106649:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.106946:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1001 00000100:00100000:1.0:1769074770.106959:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:2409:x1855007616812160:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074770.106972:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074770.106994:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd53801 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba55 expref: 236 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.107010:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074770.107015:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074770.107051:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:2409:x1855007616812160:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 94us (2032us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.107060:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1001 00000100:00100000:1.0:1769074770.115164:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.115394:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812544 00000100:00100000:2.0:1769074770.115409:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812544 00000100:00100000:2.0:1769074770.115475:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14005 00000100:00100000:2.0:1769074770.115485:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62800:x1855007616812544:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.115504:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.115523:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba63 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.115551:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 17 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.115559:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.115562:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.115565:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 30 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.115598:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 17 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.115610:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 17 pid: 10284 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074770.115625:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625127680:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.115737:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.118370:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.118499:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625127552:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074770.118525:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074770.118595:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616812032:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12195us (13607us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.118609:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6983 00000100:00100000:2.0:1769074770.118803:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812672 00000100:00100000:2.0:1769074770.118849:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812672 00002000:00100000:2.0:1769074770.118900:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2351104->2355199) req@ffff89b54cfeed80 x1855007616812672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.118911:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfeed80 with x1855007616812672 ext(2351104->2355199) 00002000:00010000:2.0:1769074770.118916:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfeed80 00000100:00100000:2.0:1769074770.118940:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8495 00000100:00100000:2.0:1769074770.118949:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616812672:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074770.119101:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616812672:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 153us (754us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074770.119108:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8495 00002000:00100000:2.0:1769074770.119114:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2351104->2355199) req@ffff89b54cfeed80 x1855007616812672/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.119122:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfeed80 with x1855007616812672 ext(2351104->2355199) 00002000:00010000:2.0:1769074770.119125:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfeed80 00000100:00100000:3.0:1769074770.133680:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.134963:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812800 00000100:00100000:2.0:1769074770.134978:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812800 00000100:00100000:2.0:1769074770.135039:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14006 00000100:00100000:2.0:1769074770.135050:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62807:x1855007616812800:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074770.135069:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.135087:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba6a expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.135120:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.135130:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 32 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.135137:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.135139:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.135143:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [536576->753663] (req 536576->753663) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 32 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.135198:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 32 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.135211:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 32 pid: 9048 timeout: 1857 lvb_type: 1 00000100:00100000:2.0:1769074770.135224:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625127808:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.135335:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.135345:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 16 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:2.0:1769074770.135366:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625127936:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.135392:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b039c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.141667:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.141770:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616812928 00000100:00100000:0.0:1769074770.141781:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616812928 00002000:00100000:0.0:1769074770.141866:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2506752->2510847) req@ffff89b54cfee680 x1855007616812928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.141878:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfee680 with x1855007616812928 ext(2506752->2510847) 00002000:00010000:0.0:1769074770.141884:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfee680 00000100:00100000:0.0:1769074770.141922:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8496 00000100:00100000:0.0:1769074770.141932:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616812928:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074770.142098:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616812928:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 166us (441us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074770.142107:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8496 00002000:00100000:0.0:1769074770.142115:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2506752->2510847) req@ffff89b54cfee680 x1855007616812928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.142123:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfee680 with x1855007616812928 ext(2506752->2510847) 00002000:00010000:0.0:1769074770.142127:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfee680 00000100:00100000:2.0:1769074770.142901:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625127680:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.142942:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.142964:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47600, rc 0) 00000100:00100000:2.0:1769074770.143024:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62800:x1855007616812544:12345-192.168.204.60@tcp:101:cat.0 Request processed in 27539us (27875us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.143033:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14005 00000100:00100000:3.0:1769074770.146113:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.146377:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.146406:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.146436:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813184 00000100:00100000:3.0:1769074770.146445:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813184 00010000:00010000:3.0:1769074770.146482:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 32 pid: 9048 timeout: 1857 lvb_type: 1 00000100:00100000:3.0:1769074770.146513:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813312 00000100:00100000:3.0:1769074770.146516:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813312 00010000:00010000:3.0:1769074770.146529:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 17 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:3.0:1769074770.146551:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6984 00000100:00100000:3.0:1769074770.146559:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616813184:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.146574:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.146592:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 32 pid: 9048 timeout: 1857 lvb_type: 1 00010000:00010000:3.0:1769074770.146604:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 32 pid: 9048 timeout: 1857 lvb_type: 1 00010000:00010000:3.0:1769074770.146627:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.146669:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616813184:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 111us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.146676:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6984 00000100:00100000:3.0:1769074770.146692:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6985 00000100:00100000:3.0:1769074770.146698:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616813312:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.146705:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.146712:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 17 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074770.146725:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 17 pid: 8396 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074770.146737:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.146739:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.146746:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 31 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.146755:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 31 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.146787:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 31 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074770.146799:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625128064:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074770.146878:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00010000:00010000:0.0:1769074770.147414:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53736 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9b4 expref: 31 pid: 9048 timeout: 1857 lvb_type: 1 00000100:00100000:0.0:1769074770.147430:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625127808:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074770.147451:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53672 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b92f expref: 17 pid: 8396 timeout: 1857 lvb_type: 0 00000100:00100000:0.0:1769074770.147460:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625127936:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074770.147484:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.147494:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46800, rc 0) 00000100:00100000:0.0:1769074770.147537:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:62807:x1855007616812800:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12490us (13871us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.147545:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14006 00000100:00100000:2.0:1769074770.147673:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813056 00000100:00100000:2.0:1769074770.147679:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813056 00002000:00100000:2.0:1769074770.147713:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1220608->1720319) req@ffff89b54cfedf80 x1855007616813056/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.147721:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfedf80 with x1855007616813056 ext(1220608->1720319) 00010000:00010000:2.0:1769074770.147729:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 17 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074770.147739:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 17 pid: 10284 timeout: 1857 lvb_type: 0 00002000:00010000:2.0:1769074770.147745:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cfedf80 00000100:00100000:2.0:1769074770.147772:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 748 00000100:00100000:2.0:1769074770.147780:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616813056:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074770.159972:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625128064:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074770.159998:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.160052:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616813312:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13353us (13645us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.160064:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6985 00000100:00100000:3.0:1769074770.176788:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616811776:12345-192.168.204.60@tcp:4:cat.0 Request processed in 106142us (106355us total) trans 21474841190 rc 0/0 00000100:00100000:3.0:1769074770.176833:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 747 00002000:00100000:3.0:1769074770.176850:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2994175) req@ffff89b54cfee300 x1855007616811776/t21474841190(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/448 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.176862:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfee300 with x1855007616811776 ext(2097152->2994175) 00010000:00010000:3.0:1769074770.176873:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:3.0:1769074770.176892:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1858 lvb_type: 0 00002000:00010000:3.0:1769074770.176901:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54cfee300 00000100:00100000:0.0:1769074770.177339:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.177989:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813440 00000100:00100000:3.0:1769074770.177999:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813440 00002000:00100000:3.0:1769074770.178036:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (536576->540671) req@ffff89b54759b480 x1855007616813440/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.178045:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759b480 with x1855007616813440 ext(536576->540671) 00002000:00010000:3.0:1769074770.178050:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759b480 00000100:00100000:3.0:1769074770.178076:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8497 00000100:00100000:3.0:1769074770.178084:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616813440:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074770.178242:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616813440:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 158us (905us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074770.178249:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8497 00002000:00100000:3.0:1769074770.178255:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (536576->540671) req@ffff89b54759b480 x1855007616813440/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.178263:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759b480 with x1855007616813440 ext(536576->540671) 00002000:00010000:3.0:1769074770.178266:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759b480 00000100:00100000:0.0:1769074770.179138:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.179281:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813568 00000100:00100000:3.0:1769074770.179288:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813568 00010000:00010000:3.0:1769074770.179315:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074770.179361:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6986 00000100:00100000:3.0:1769074770.179369:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616813568:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.179381:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.179399:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.179409:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.179432:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47e00/0x7fe2323adfd53767 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2994175) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9ec expref: 28 pid: 10284 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.179462:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.179464:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.179471:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.179481:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.179515:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074770.179533:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625128192:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074770.179604:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.186757:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625128192:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074770.186787:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.186887:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616813568:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7517us (9447us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.186899:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6986 00000100:00100000:1.0:1769074770.187539:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.187619:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813696 00000100:00100000:0.0:1769074770.187628:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813696 00002000:00100000:0.0:1769074770.187674:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (749568->753663) req@ffff89b546f94380 x1855007616813696/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.187684:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94380 with x1855007616813696 ext(749568->753663) 00002000:00010000:0.0:1769074770.187690:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94380 00000100:00100000:0.0:1769074770.187729:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8498 00000100:00100000:0.0:1769074770.187737:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616813696:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074770.187911:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616813696:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 174us (373us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074770.187918:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8498 00002000:00100000:0.0:1769074770.187924:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (749568->753663) req@ffff89b546f94380 x1855007616813696/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.187931:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94380 with x1855007616813696 ext(749568->753663) 00002000:00010000:0.0:1769074770.187935:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94380 00000100:00100000:2.0:1769074770.189404:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616813056:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 41616us (43296us total) trans 17179874503 rc 0/0 00000100:00100000:2.0:1769074770.189417:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 748 00002000:00100000:2.0:1769074770.189428:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1220608->1720319) req@ffff89b54cfedf80 x1855007616813056/t17179874503(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/448 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.189437:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfedf80 with x1855007616813056 ext(1220608->1720319) 00010000:00010000:2.0:1769074770.189446:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 18 pid: 10284 timeout: 1857 lvb_type: 0 00010000:00010000:2.0:1769074770.189464:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 18 pid: 10284 timeout: 1858 lvb_type: 0 00002000:00010000:2.0:1769074770.189476:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54cfedf80 00000100:00100000:1.0:1769074770.203775:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.203909:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.204270:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.204402:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814080 00000100:00100000:3.0:1769074770.204418:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814080 00010000:00010000:3.0:1769074770.204465:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 18 pid: 10284 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074770.204511:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6987 00000100:00100000:3.0:1769074770.204520:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616814080:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.204537:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.204560:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 18 pid: 10284 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.204574:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 18 pid: 10284 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.204603:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47400/0x7fe2323adfd5376e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1220608->1720319) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b9f3 expref: 18 pid: 10284 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.204628:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.204630:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.204638:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.204648:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.204685:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074770.204710:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625128320:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074770.204789:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074770.207607:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813952 00000100:00100000:2.0:1769074770.207627:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813952 00002000:00100000:2.0:1769074770.207694:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b546f96680 x1855007616813952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.207719:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96680 with x1855007616813952 ext(3145728->4194303) 00002000:00010000:2.0:1769074770.207725:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f96680 00000100:00100000:2.0:1769074770.207757:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616813824 00000100:00100000:2.0:1769074770.207760:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616813824 00002000:00100000:2.0:1769074770.207776:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546f95500 x1855007616813824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.207783:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95500 with x1855007616813824 ext(2097152->4194303) 00002000:00010000:2.0:1769074770.207787:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f95500 00000100:00100000:2.0:1769074770.207830:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8499 00000100:00100000:2.0:1769074770.207839:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2409:x1855007616813952:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074770.208672:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8500 00000100:00100000:3.0:1769074770.208683:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616813824:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074770.218478:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625128320:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074770.218509:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074770.218581:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616814080:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 14061us (14310us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.218592:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6987 00000100:00100000:1.0:1769074770.223964:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.224161:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814208 00000100:00100000:3.0:1769074770.224175:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814208 00002000:00100000:3.0:1769074770.224235:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546f94e00 x1855007616814208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.224248:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94e00 with x1855007616814208 ext(1048576->2097151) 00002000:00010000:3.0:1769074770.224257:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94e00 00000100:00100000:3.0:1769074770.224339:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8501 00000100:00100000:3.0:1769074770.224361:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616814208:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074770.231999:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616813952:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 24145us (28219us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074770.232016:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8499 00002000:00100000:1.0:1769074770.232030:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b546f96680 x1855007616813952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074770.232042:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f96680 with x1855007616813952 ext(3145728->4194303) 00002000:00010000:1.0:1769074770.232048:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f96680 00000100:00100000:0.0:1769074770.247110:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616813824:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 38418us (43188us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074770.247125:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8500 00002000:00100000:0.0:1769074770.247137:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546f95500 x1855007616813824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.247147:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f95500 with x1855007616813824 ext(2097152->4194303) 00002000:00010000:0.0:1769074770.247153:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f95500 00000100:00100000:3.0:1769074770.252900:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616814208:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 28543us (28941us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074770.252911:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8501 00002000:00100000:3.0:1769074770.252921:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546f94e00 x1855007616814208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.252928:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f94e00 with x1855007616814208 ext(1048576->2097151) 00002000:00010000:3.0:1769074770.252933:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f94e00 00000100:00100000:2.0:1769074770.264346:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.264398:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.264494:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.265325:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814336 00000100:00100000:1.0:1769074770.265340:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814336 00000100:00100000:1.0:1769074770.265477:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814464 00000100:00100000:1.0:1769074770.265482:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814464 00000100:00100000:1.0:1769074770.265506:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814592 00000100:00100000:1.0:1769074770.265509:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814592 00000100:00100000:1.0:1769074770.265537:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14007 00000100:00100000:1.0:1769074770.265549:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616814336:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.265570:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.265592:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53824 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba71 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074770.265668:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128448:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074770.265762:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.265786:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14008 00000100:00100000:1.0:1769074770.265794:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616814464:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.265803:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.265836:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba78 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.265850:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074770.265853:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074770.265863:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 22 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.265881:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 22 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.265895:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:1.0:1769074770.265929:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616814464:12345-192.168.204.60@tcp:101:cat.0 Request processed in 135us (1529us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.265937:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14008 00000100:00100000:1.0:1769074770.265966:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14009 00000100:00100000:1.0:1769074770.265973:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616814592:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.265981:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.265990:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba7f expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.266002:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074770.266004:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074770.266009:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 20 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.266020:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 20 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.266029:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052000, rc 0) 00000100:00100000:1.0:1769074770.266049:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616814592:12345-192.168.204.60@tcp:101:cat.0 Request processed in 77us (1556us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.266055:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14009 00002000:00010000:1.0:1769074770.275370:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074768 -> 1769074769 00002000:00010000:1.0:1769074770.275379:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074768 -> 1769074769 00002000:00010000:1.0:1769074770.275382:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074768 -> 1769074769 00000100:00100000:2.0:1769074770.275388:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.275401:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128448:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074770.275437:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53824 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba71 expref: 19 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.275458:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53824 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba71 expref: 19 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.275470:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:1.0:1769074770.275537:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616814336:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9990us (11203us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074770.275545:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14007 00000100:00100000:1.0:1769074770.275572:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814720 00000100:00100000:2.0:1769074770.275575:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.275578:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814720 00000100:00100000:2.0:1769074770.275599:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814848 00000100:00100000:2.0:1769074770.275602:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814848 00000100:00100000:1.0:1769074770.275616:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14010 00000100:00100000:1.0:1769074770.275623:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616814720:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.275641:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.275656:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53839 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba86 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074770.275702:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128576:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074770.275753:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.278610:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128576:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074770.278650:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53839 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba86 expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.278671:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53839 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba86 expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.278685:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:1.0:1769074770.278746:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616814720:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3121us (3361us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074770.284570:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14011 00000100:00100000:2.0:1769074770.284599:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616814848:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.284619:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.284640:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba8d expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.284666:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.284668:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.284677:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 28 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.284705:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 28 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.284718:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602b800, rc 0) 00000100:00100000:2.0:1769074770.285719:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616814848:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1129us (10135us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.285730:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14011 00000100:00100000:1.0:1769074770.285792:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14010 00000100:00100000:0.0:1769074770.288589:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.288690:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616814976 00000100:00100000:3.0:1769074770.288713:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616814976 00002000:00100000:3.0:1769074770.288768:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5446c2d80 x1855007616814976/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.288778:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c2d80 with x1855007616814976 ext(4194304->4808703) 00002000:00010000:3.0:1769074770.288785:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c2d80 00000100:00100000:3.0:1769074770.288840:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8502 00000100:00100000:3.0:1769074770.288850:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616814976:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074770.298547:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.303495:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815104 00000100:00100000:2.0:1769074770.303512:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815104 00000100:00100000:2.0:1769074770.303646:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14012 00000100:00100000:2.0:1769074770.303659:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:62800:x1855007616815104:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.303676:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.303708:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba94 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.303738:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.303747:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.303749:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.303753:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.303787:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 19 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.303800:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 19 pid: 9048 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.303839:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128704:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.303939:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.307106:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616814976:12345-192.168.204.60@tcp:3:cat.0 Request processed in 18242us (18514us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074770.307123:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8502 00002000:00100000:1.0:1769074770.307136:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5446c2d80 x1855007616814976/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074770.307145:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c2d80 with x1855007616814976 ext(4194304->4808703) 00010000:00010000:1.0:1769074770.307155:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 18 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:1.0:1769074770.307173:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 18 pid: 9048 timeout: 1858 lvb_type: 0 00002000:00010000:1.0:1769074770.307181:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5446c2d80 00000100:00100000:2.0:1769074770.308182:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128704:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.308633:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.308653:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602ae00, rc 0) 00000100:00100000:2.0:1769074770.308713:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62800:x1855007616815104:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5055us (10177us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.308723:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14012 00000100:00100000:3.0:1769074770.312479:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.317146:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.317671:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815360 00000100:00100000:2.0:1769074770.317687:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815360 00000100:00100000:2.0:1769074770.317765:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14013 00000100:00100000:2.0:1769074770.317776:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616815360:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.317797:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.317855:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd5384e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ba9b expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074770.317928:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128832:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074770.318021:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074770.318110:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.318232:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815232 00000100:00100000:1.0:1769074770.318239:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815232 00002000:00100000:1.0:1769074770.318273:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2351104->2510847) req@ffff89b54510bb80 x1855007616815232/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074770.318283:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510bb80 with x1855007616815232 ext(2351104->2510847) 00010000:00010000:1.0:1769074770.318290:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 20 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:1.0:1769074770.318302:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 20 pid: 9048 timeout: 1858 lvb_type: 0 00002000:00010000:1.0:1769074770.318309:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54510bb80 00000100:00100000:1.0:1769074770.318329:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 749 00000100:00100000:1.0:1769074770.318336:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616815232:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074770.318823:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815488 00000100:00100000:2.0:1769074770.318830:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815488 00000100:00100000:2.0:1769074770.318877:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14014 00000100:00100000:2.0:1769074770.318885:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616815488:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.318894:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.318908:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea1400/0x7fe2323adfd53855 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879baa2 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074770.318949:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625128960:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074770.318994:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.327200:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625128832:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074770.327245:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd5384e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba9b expref: 23 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074770.327271:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd5384e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba9b expref: 23 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074770.327285:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea0400, rc 0) 00000100:00100000:3.0:1769074770.327362:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616815360:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9586us (10224us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074770.327373:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14013 00000100:00100000:2.0:1769074770.329611:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625128960:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074770.329658:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea1400/0x7fe2323adfd53855 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa2 expref: 21 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.329683:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea1400/0x7fe2323adfd53855 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa2 expref: 21 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.329712:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea1400, rc 0) 00000100:00100000:2.0:1769074770.329764:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616815488:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10879us (11653us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074770.329772:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14014 00000100:00100000:1.0:1769074770.332722:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616815232:12345-192.168.204.60@tcp:4:cat.0 Request processed in 14371us (20245us total) trans 17179874504 rc 0/0 00000100:00100000:1.0:1769074770.332739:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 749 00002000:00100000:1.0:1769074770.332754:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2351104->2510847) req@ffff89b54510bb80 x1855007616815232/t17179874504(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/448 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074770.332765:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510bb80 with x1855007616815232 ext(2351104->2510847) 00010000:00010000:1.0:1769074770.332776:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 20 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:1.0:1769074770.332792:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 20 pid: 9048 timeout: 1858 lvb_type: 0 00002000:00010000:1.0:1769074770.332800:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54510bb80 00000100:00100000:0.0:1769074770.334627:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:3.0:1769074770.334681:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074770.334843:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815616 00000100:00100000:1.0:1769074770.334854:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815616 00000100:00100000:1.0:1769074770.334913:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14015 00000100:00100000:1.0:1769074770.334924:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62802:x1855007616815616:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.334946:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.334964:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879baa9 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.334991:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074770.334993:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074770.334998:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4804608->5242879] (req 4804608->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.335015:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4804608->5242879] (req 4804608->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.335026:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:1.0:1769074770.335080:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62802:x1855007616815616:12345-192.168.204.60@tcp:101:cat.0 Request processed in 158us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.335087:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14015 00000100:00100000:0.0:1769074770.342379:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.342489:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815744 00000100:00100000:2.0:1769074770.342501:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815744 00010000:00010000:2.0:1769074770.342542:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 21 pid: 9048 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.342644:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6988 00000100:00100000:2.0:1769074770.342655:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616815744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074770.342671:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074770.342696:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 21 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.342710:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 21 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.342733:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46600/0x7fe2323adfd5380f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2351104->2510847) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba5c expref: 21 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.342751:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.342753:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.342761:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 30 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.342777:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 30 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.342786:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.342788:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.342791:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.342798:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.342883:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074770.342897:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625129088:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074770.342988:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 30 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074770.342997:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625129216:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074770.343019:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.354880:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.355005:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625129088:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074770.355019:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625129216:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074770.355052:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074770.355106:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616815744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12453us (12737us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.355118:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6988 00000100:00100000:3.0:1769074770.355413:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616815872 00000100:00100000:3.0:1769074770.355422:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616815872 00000100:00100000:3.0:1769074770.355535:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14016 00000100:00100000:3.0:1769074770.355544:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616815872:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074770.355561:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074770.355580:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd53863 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bab0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074770.355633:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625129344:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074770.355702:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.371364:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625129344:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074770.371411:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd53863 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bab0 expref: 21 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074770.371439:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd53863 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bab0 expref: 21 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074770.371455:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9000, rc 0) 00000100:00100000:3.0:1769074770.371540:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616815872:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15996us (16670us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074770.371549:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14016 00000100:00100000:0.0:1769074770.374468:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.374521:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.374553:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.374588:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.374622:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816000 00000100:00100000:0.0:1769074770.374637:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816000 00000100:00100000:0.0:1769074770.374698:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 329 00000100:00100000:0.0:1769074770.374709:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b09850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2406:x1855007616816000:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074770.374797:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b09850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2406:x1855007616816000:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 89us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.374843:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 329 00000100:00100000:0.0:1769074770.374901:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.375021:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.375043:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.375065:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.375164:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816128 00000100:00100000:1.0:1769074770.375170:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816128 00000100:00100000:1.0:1769074770.375219:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816384 00000100:00100000:1.0:1769074770.375223:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816384 00000100:00100000:1.0:1769074770.375235:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816640 00000100:00100000:1.0:1769074770.375237:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816640 00000100:00100000:1.0:1769074770.375252:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816256 00000100:00100000:1.0:1769074770.375254:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816256 00000100:00100000:1.0:1769074770.375268:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816512 00000100:00100000:1.0:1769074770.375270:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816512 00000100:00100000:1.0:1769074770.375284:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2307 00000100:00100000:1.0:1769074770.375291:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616816128:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375337:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616816128:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 46us (880us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375342:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2307 00000100:00100000:1.0:1769074770.375374:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2308 00000100:00100000:1.0:1769074770.375381:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2407:x1855007616816384:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375407:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2407:x1855007616816384:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (885us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375413:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2308 00000100:00100000:1.0:1769074770.375424:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2309 00000100:00100000:1.0:1769074770.375430:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616816640:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375452:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616816640:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 22us (899us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375458:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2309 00000100:00100000:1.0:1769074770.375470:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2310 00000100:00100000:1.0:1769074770.375476:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616816256:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375505:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616816256:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375511:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2310 00000100:00100000:1.0:1769074770.375523:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2311 00000100:00100000:1.0:1769074770.375528:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2406:x1855007616816512:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375557:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2406:x1855007616816512:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375562:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2311 00000100:00100000:1.0:1769074770.375586:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816768 00000100:00100000:1.0:1769074770.375589:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816768 00000100:00100000:1.0:1769074770.375613:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616816896 00000100:00100000:1.0:1769074770.375616:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616816896 00000100:00100000:1.0:1769074770.375639:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1442 00000100:00100000:1.0:1769074770.375644:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2406:x1855007616816768:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375669:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2406:x1855007616816768:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (626us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375676:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1442 00000100:00100000:1.0:1769074770.375688:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1443 00000100:00100000:1.0:1769074770.375693:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616816896:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074770.375718:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616816896:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.375723:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1443 00000100:00080000:0.0:1769074770.376042:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074770.376049:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074770.376051:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074770.376053:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074770.379924:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.379955:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.379999:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.380094:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616817152 00000100:00100000:2.0:1769074770.380106:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616817152 00002000:00100000:2.0:1769074770.380159:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4804608->4808703) req@ffff89b5461e5880 x1855007616817152/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.380170:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5880 with x1855007616817152 ext(4804608->4808703) 00002000:00010000:2.0:1769074770.380177:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e5880 00000100:00100000:2.0:1769074770.380207:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616817024 00000100:00100000:2.0:1769074770.380210:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616817024 00002000:00100000:2.0:1769074770.380234:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5461e5500 x1855007616817024/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.380241:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5500 with x1855007616817024 ext(2097152->4194303) 00002000:00010000:2.0:1769074770.380245:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e5500 00000100:00100000:2.0:1769074770.380261:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616817280 00000100:00100000:2.0:1769074770.380264:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616817280 00002000:00100000:2.0:1769074770.380277:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5461e7b80 x1855007616817280/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.380285:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7b80 with x1855007616817280 ext(2097152->4194303) 00002000:00010000:2.0:1769074770.380288:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5461e7b80 00000100:00100000:2.0:1769074770.380304:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8503 00000100:00100000:2.0:1769074770.380313:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616817152:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074770.380514:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616817152:12345-192.168.204.60@tcp:3:cat.0 Request processed in 200us (599us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074770.380522:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8503 00002000:00100000:2.0:1769074770.380531:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4804608->4808703) req@ffff89b5461e5880 x1855007616817152/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.380539:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5880 with x1855007616817152 ext(4804608->4808703) 00002000:00010000:2.0:1769074770.380542:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e5880 00000100:00100000:2.0:1769074770.380560:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8504 00000100:00100000:2.0:1769074770.380567:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616817024:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074770.381755:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8505 00000100:00100000:2.0:1769074770.381769:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616817280:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074770.427975:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616817024:12345-192.168.204.60@tcp:3:cat.0 Request processed in 47396us (48006us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074770.427992:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8504 00002000:00100000:2.0:1769074770.428008:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5461e5500 x1855007616817024/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.428021:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5500 with x1855007616817024 ext(2097152->4194303) 00002000:00010000:2.0:1769074770.428029:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e5500 00000100:00100000:2.0:1769074770.452750:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616817280:12345-192.168.204.60@tcp:3:cat.0 Request processed in 70973us (72737us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074770.452765:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8505 00002000:00100000:2.0:1769074770.452782:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5461e7b80 x1855007616817280/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.452793:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7b80 with x1855007616817280 ext(2097152->4194303) 00002000:00010000:2.0:1769074770.452800:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5461e7b80 00000100:00100000:3.0:1769074770.485123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.485255:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.485475:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616817664 00000100:00100000:1.0:1769074770.485487:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616817664 00000100:00100000:1.0:1769074770.485550:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13312 00000100:00100000:1.0:1769074770.485560:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62815:x1855007616817664:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074770.485586:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.485608:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5386a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bab7 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.485708:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.485725:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00000100:00100000:2.0:1769074770.485728:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616817408 00000100:00100000:2.0:1769074770.485733:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616817408 00010000:00010000:1.0:1769074770.485736:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.485749:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.485756:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.485766:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00002000:00100000:2.0:1769074770.485768:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54510ad80 x1855007616817408/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.485776:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510ad80 with x1855007616817408 ext(4194304->4808703) 00002000:00010000:2.0:1769074770.485782:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54510ad80 00000100:00100000:2.0:1769074770.485834:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8506 00000100:00100000:2.0:1769074770.485841:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2407:x1855007616817408:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:1.0:1769074770.485910:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5387f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.485921:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5387f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.485928:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5387f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074770.485937:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074770.486015:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486023:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486028:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486033:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486039:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910da00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074770.486043:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486049:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486061:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074770.486064:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074770.486068:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486074:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53871 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486088:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486093:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486098:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486102:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910ce00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074770.486104:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486109:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486115:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074770.486117:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074770.486120:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486124:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53878 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074770.486169:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d200/0x7fe2323adfd5386a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bab7 expref: 238 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.486184:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5387f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bab7 expref: 237 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.486191:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074770.486193:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074770.486196:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c600, rc 0) 00000100:00100000:1.0:1769074770.486252:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62815:x1855007616817664:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 693us (995us total) trans 8589943931 rc 0/0 00000100:00100000:1.0:1769074770.486260:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13312 00000100:00100000:3.0:1769074770.496962:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.497030:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.497292:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616817792 00000100:00100000:0.0:1769074770.497304:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616817792 00000100:00100000:0.0:1769074770.497381:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1002 00000100:00100000:0.0:1769074770.497391:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:2408:x1855007616817792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074770.497410:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074770.497440:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c600/0x7fe2323adfd5387f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bab7 expref: 237 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.497459:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074770.497465:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074770.497523:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:2408:x1855007616817792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 133us (571us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.497530:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1002 00000100:00100000:1.0:1769074770.497851:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818048 00000100:00100000:1.0:1769074770.497858:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818048 00000100:00100000:1.0:1769074770.497898:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4660 00000100:00100000:1.0:1769074770.497906:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:62815:x1855007616818048:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074770.498159:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2407:x1855007616817408:12345-192.168.204.60@tcp:3:cat.0 Request processed in 12316us (13049us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074770.499418:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8506 00002000:00100000:2.0:1769074770.499438:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54510ad80 x1855007616817408/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1769074770.499447:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62815:x1855007616818048:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 1530us (2405us total) trans 8589943932 rc 0/0 00002000:00100000:2.0:1769074770.499449:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510ad80 with x1855007616817408 ext(4194304->4808703) 00002000:00010000:2.0:1769074770.499456:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54510ad80 00000100:00100000:1.0:1769074770.499458:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4660 00000100:00100000:3.0:1769074770.538629:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.538753:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818176 00000100:00100000:0.0:1769074770.538764:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818176 00000100:00100000:0.0:1769074770.538867:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14017 00000100:00100000:0.0:1769074770.538879:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62815:x1855007616818176:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074770.538902:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074770.538926:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879babe expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.538962:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.538973:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 19 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074770.538981:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074770.538983:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074770.538988:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1298432->1605631] (req 1298432->1605631) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.539025:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 19 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074770.539045:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 19 pid: 10284 timeout: 1858 lvb_type: 1 00000100:00100000:0.0:1769074770.539060:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625129472:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074770.539155:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.539165:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 1858 lvb_type: 0 00000100:00100000:0.0:1769074770.539174:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625129600:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074770.539202:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.544615:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625129472:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074770.544633:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625129600:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074770.544667:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1298432->1605631] (req 1298432->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.544687:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:3.0:1769074770.544753:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62815:x1855007616818176:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 5876us (6134us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.544763:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14017 00000100:00100000:1.0:1769074770.550705:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.550749:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.550831:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818304 00000100:00100000:3.0:1769074770.550841:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818304 00010000:00010000:3.0:1769074770.550881:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 20 pid: 10284 timeout: 1858 lvb_type: 1 00000100:00100000:3.0:1769074770.550942:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818432 00000100:00100000:3.0:1769074770.550949:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818432 00010000:00010000:3.0:1769074770.550969:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074770.551021:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6989 00000100:00100000:3.0:1769074770.551035:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616818304:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.551054:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.551081:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 20 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:3.0:1769074770.551097:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 20 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:3.0:1769074770.551127:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052000/0x7fe2323adfd53832 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba7f expref: 20 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:3.0:1769074770.551151:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.551224:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616818304:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 193us (530us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.551232:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6989 00000100:00100000:3.0:1769074770.551252:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6990 00000100:00100000:3.0:1769074770.551258:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616818432:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.551266:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.551274:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.551284:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.551300:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47600/0x7fe2323adfd53816 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba63 expref: 32 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.551310:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.551313:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.551321:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.551331:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.551384:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074770.551400:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625129728:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074770.551462:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.555362:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.555498:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818560 00000100:00100000:2.0:1769074770.555509:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818560 00002000:00100000:2.0:1769074770.555573:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1298432->1302527) req@ffff89b5437b6300 x1855007616818560/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.555585:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6300 with x1855007616818560 ext(1298432->1302527) 00002000:00010000:2.0:1769074770.555592:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b6300 00000100:00100000:2.0:1769074770.555652:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8507 00000100:00100000:2.0:1769074770.555666:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616818560:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074770.555949:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616818560:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 284us (608us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074770.555963:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8507 00002000:00100000:2.0:1769074770.555973:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1298432->1302527) req@ffff89b5437b6300 x1855007616818560/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.555982:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6300 with x1855007616818560 ext(1298432->1302527) 00002000:00010000:2.0:1769074770.555986:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b6300 00000100:00100000:3.0:1769074770.556767:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625129728:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074770.556796:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.556876:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616818432:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5615us (6124us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.556887:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6990 00000100:00100000:1.0:1769074770.573111:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.573535:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818688 00000100:00100000:0.0:1769074770.573554:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818688 00002000:00100000:0.0:1769074770.573632:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1601536->1605631) req@ffff89b5437b4000 x1855007616818688/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.573646:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4000 with x1855007616818688 ext(1601536->1605631) 00002000:00010000:0.0:1769074770.573653:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4000 00000100:00100000:0.0:1769074770.573727:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8508 00000100:00100000:0.0:1769074770.573744:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616818688:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074770.574065:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616818688:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 324us (962us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074770.574076:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8508 00002000:00100000:0.0:1769074770.574086:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1601536->1605631) req@ffff89b5437b4000 x1855007616818688/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.574095:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4000 with x1855007616818688 ext(1601536->1605631) 00002000:00010000:0.0:1769074770.574100:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4000 00000100:00100000:0.0:1769074770.673048:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.673202:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.673666:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818816 00000100:00100000:3.0:1769074770.673685:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818816 00000100:00100000:1.0:1769074770.673936:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616818944 00000100:00100000:1.0:1769074770.673943:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616818944 00000100:00100000:1.0:1769074770.674006:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14018 00000100:00100000:1.0:1769074770.674020:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616818816:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.674043:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.674066:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd53894 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bac5 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074770.674142:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625129856:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074770.674374:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074770.674512:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14019 00000100:00100000:0.0:1769074770.674522:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616818944:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074770.674531:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074770.674543:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd5389b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bacc expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074770.674572:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625129984:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074770.674602:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00002000:00010000:3.0:1769074770.678450:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074769 -> 1769074770 00002000:00010000:3.0:1769074770.678465:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074769 -> 1769074770 00002000:00010000:3.0:1769074770.678468:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074769 -> 1769074770 00000100:00100000:3.0:1769074770.678910:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625129856:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074770.678970:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd53894 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bac5 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074770.679003:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d800/0x7fe2323adfd53894 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bac5 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074770.679026:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d800, rc 0) 00000100:00100000:3.0:1769074770.679127:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616818816:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5109us (6090us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074770.679138:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14018 00002000:00010000:2.0:1769074770.679292:0:28954:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074769 -> 1769074770 00002000:00010000:2.0:1769074770.679298:0:28954:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074769 -> 1769074770 00002000:00010000:2.0:1769074770.679300:0:28954:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074769 -> 1769074770 00000100:00100000:2.0:1769074770.679314:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625129984:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074770.679338:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd5389b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bacc expref: 20 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.679366:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd5389b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bacc expref: 20 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.679377:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e400, rc 0) 00000100:00100000:2.0:1769074770.679413:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616818944:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4891us (6211us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074770.679421:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14019 00000100:00100000:2.0:1769074770.684639:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.684971:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819072 00000100:00100000:0.0:1769074770.684982:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819072 00000100:00100000:0.0:1769074770.685073:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14020 00000100:00100000:0.0:1769074770.685084:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62820:x1855007616819072:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074770.685106:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074770.685129:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bad3 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.685425:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.685437:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074770.685498:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074770.685503:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.685542:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 30 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.685557:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 30 pid: 28954 timeout: 1858 lvb_type: 0 00000100:00100000:0.0:1769074770.685572:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625130112:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074770.688015:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.697087:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.697170:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819200 00000100:00100000:2.0:1769074770.697180:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819200 00002000:00100000:2.0:1769074770.697228:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (536576->753663) req@ffff89b5437b6a00 x1855007616819200/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.697246:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6a00 with x1855007616819200 ext(536576->753663) 00010000:00010000:2.0:1769074770.697255:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 31 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.697272:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 31 pid: 28954 timeout: 1858 lvb_type: 0 00002000:00010000:2.0:1769074770.697280:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5437b6a00 00000100:00100000:2.0:1769074770.697316:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 750 00000100:00100000:2.0:1769074770.697327:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616819200:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074770.705915:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616819200:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 8567us (8815us total) trans 17179874505 rc 0/0 00000100:00100000:2.0:1769074770.705937:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 750 00002000:00100000:2.0:1769074770.705951:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (536576->753663) req@ffff89b5437b6a00 x1855007616819200/t17179874505(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/448 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.705965:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6a00 with x1855007616819200 ext(536576->753663) 00010000:00010000:2.0:1769074770.705978:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 33 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.706001:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 33 pid: 28954 timeout: 1858 lvb_type: 0 00002000:00010000:2.0:1769074770.706011:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5437b6a00 00000100:00100000:0.0:1769074770.708034:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625130112:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074770.708073:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.708096:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:2.0:1769074770.708724:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.708904:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819328 00000100:00100000:3.0:1769074770.708913:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819328 00010000:00010000:3.0:1769074770.708954:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 33 pid: 28954 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074770.708994:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6991 00000100:00100000:3.0:1769074770.709004:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616819328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.709017:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.709037:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 33 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.709049:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 33 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.709074:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e46800/0x7fe2323adfd5381d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 536576->753663) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba6a expref: 33 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.709090:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.709092:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074770.709099:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.709116:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.709148:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074770.709173:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625130240:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074770.709232:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074770.709587:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62820:x1855007616819072:12345-192.168.204.60@tcp:101:cat.0 Request processed in 23077us (23531us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.709600:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14020 00000100:00100000:3.0:1769074770.717563:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625130240:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074770.717591:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.717655:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616819328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8652us (8932us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.717666:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6991 00000100:00100000:3.0:1769074770.718127:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.718615:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819456 00000100:00100000:2.0:1769074770.718623:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819456 00000100:00100000:2.0:1769074770.718668:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14021 00000100:00100000:2.0:1769074770.718677:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62814:x1855007616819456:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074770.718705:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.718724:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bada expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.718755:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 27 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.718765:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.718773:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.718775:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.718779:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1310720->2097151] (req 1310720->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 23 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.718843:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074770.718857:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00000100:00100000:2.0:1769074770.718870:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625130368:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.718986:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 27 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.718997:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 27 pid: 8398 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.719006:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625130496:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.719030:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c510c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074770.731911:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625130368:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074770.734093:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.734131:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.734265:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819712 00000100:00100000:0.0:1769074770.734276:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819712 00002000:00100000:0.0:1769074770.734329:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5437b6680 x1855007616819712/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.734352:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6680 with x1855007616819712 ext(0->1048575) 00002000:00010000:0.0:1769074770.734360:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b6680 00000100:00100000:0.0:1769074770.734391:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8509 00000100:00100000:0.0:1769074770.734402:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616819712:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074770.736279:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819584 00000100:00100000:3.0:1769074770.736294:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819584 00010000:00010000:3.0:1769074770.736336:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 28 pid: 8398 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074770.736399:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6992 00000100:00100000:3.0:1769074770.736410:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616819584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.736425:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.736457:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 28 pid: 8398 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.736476:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 28 pid: 8398 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.736505:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.736556:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616819584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 148us (2476us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.736564:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6992 00000100:00100000:1.0:1769074770.736718:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074770.736744:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616819840 00000100:00100000:1.0:1769074770.736750:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616819840 00002000:00100000:1.0:1769074770.736780:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5437b5f80 x1855007616819840/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074770.736791:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5f80 with x1855007616819840 ext(1048576->2097151) 00010000:00010000:1.0:1769074770.736798:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 24 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:1.0:1769074770.736833:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 24 pid: 10284 timeout: 1858 lvb_type: 1 00002000:00010000:1.0:1769074770.736842:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b5f80 00000100:00100000:1.0:1769074770.736860:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 751 00000100:00100000:1.0:1769074770.736868:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2408:x1855007616819840:12345-192.168.204.60@tcp:3:multiop.0 00010000:00010000:2.0:1769074770.739368:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437e00/0x7fe2323adfd537c2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba2b expref: 27 pid: 8398 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.739394:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625130496:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.739447:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1310720->2097151] (req 1310720->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 25 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.739468:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b08e800, rc 0) 00000100:00100000:2.0:1769074770.739505:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+25:62814:x1855007616819456:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20828us (21378us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074770.739515:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14021 00000100:00100000:2.0:1769074770.771577:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616819712:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 37159us (37428us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074770.771594:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8509 00002000:00100000:2.0:1769074770.771611:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5437b6680 x1855007616819712/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.771625:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6680 with x1855007616819712 ext(0->1048575) 00002000:00010000:2.0:1769074770.771633:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b6680 00000100:00100000:0.0:1769074770.774557:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.775360:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616819840:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 38474us (38623us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074770.775376:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 751 00002000:00100000:2.0:1769074770.775390:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5437b5f80 x1855007616819840/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.775401:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5f80 with x1855007616819840 ext(1048576->2097151) 00010000:00010000:2.0:1769074770.775412:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:2.0:1769074770.775431:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00002000:00010000:2.0:1769074770.775439:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b5f80 00000100:00100000:1.0:1769074770.777256:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820096 00000100:00100000:1.0:1769074770.777275:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820096 00000100:00100000:2.0:1769074770.777512:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13313 00000100:00100000:2.0:1769074770.777523:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62823:x1855007616820096:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.777551:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.777573:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd538b0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bae1 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.777651:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777665:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777677:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777704:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777711:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777718:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777865:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538c5 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777877:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538c5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777885:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538c5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074770.777895:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074770.777977:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777986:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777992:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.777999:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778004:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074770.778008:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778016:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778028:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074770.778031:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074770.778037:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778054:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538b7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778068:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778074:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778079:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778085:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074770.778088:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778094:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778101:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4000) 00010000:00010000:2.0:1769074770.778103:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074770.778106:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778111:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538be lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.778151:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08fa00/0x7fe2323adfd538b0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae1 expref: 240 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.778177:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538c5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae1 expref: 239 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.778184:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074770.778187:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074770.778189:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4e00, rc 0) 00000100:00100000:2.0:1769074770.778265:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:62823:x1855007616820096:12345-192.168.204.60@tcp:101:cat.0 Request processed in 743us (3722us total) trans 8589943933 rc 0/0 00000100:00100000:2.0:1769074770.778274:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13313 00000100:00100000:0.0:1769074770.790167:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.790261:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.790297:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.790352:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820608 00000100:00100000:0.0:1769074770.790366:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820608 00000100:00100000:0.0:1769074770.790441:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820224 00000100:00100000:0.0:1769074770.790444:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820224 00010000:00010000:0.0:1769074770.790464:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00000100:00100000:0.0:1769074770.790494:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6993 00000100:00100000:0.0:1769074770.790504:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2406:x1855007616820224:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074770.790521:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074770.790548:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:0.0:1769074770.790562:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:0.0:1769074770.790590:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5382b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba78 expref: 23 pid: 10284 timeout: 1858 lvb_type: 1 00010000:00010000:0.0:1769074770.790606:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074770.790608:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074770.790617:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.790628:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.790672:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074770.790688:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625130624:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074770.790777:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074770.790801:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1003 00000100:00100000:0.0:1769074770.790830:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66561cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:2408:x1855007616820608:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074770.790850:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074770.790867:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538c5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae1 expref: 239 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.790879:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074770.790885:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074770.790911:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66561cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+238:2408:x1855007616820608:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 103us (648us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.790918:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1003 00000100:00100000:3.0:1769074770.796592:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625130624:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074770.796627:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.796697:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616820224:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6185us (6389us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.796710:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6993 00000100:00100000:1.0:1769074770.797615:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820480 00000100:00100000:1.0:1769074770.797626:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820480 00000100:00100000:1.0:1769074770.797681:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4661 00000100:00100000:1.0:1769074770.797700:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c5850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+238:62823:x1855007616820480:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074770.798254:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c5850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:62823:x1855007616820480:12345-192.168.204.60@tcp:35:cat.0 Request processed in 552us (8110us total) trans 8589943934 rc 0/0 00000100:00100000:1.0:1769074770.798263:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4661 00000100:00100000:0.0:1769074770.806336:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.810187:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820736 00000100:00100000:2.0:1769074770.810212:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820736 00002000:00100000:2.0:1769074770.810304:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1310720->1314815) req@ffff89b5461e6300 x1855007616820736/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.810318:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e6300 with x1855007616820736 ext(1310720->1314815) 00002000:00010000:2.0:1769074770.810327:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5461e6300 00000100:00100000:2.0:1769074770.810449:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8510 00000100:00100000:2.0:1769074770.810459:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616820736:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074770.810680:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616820736:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 222us (4356us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074770.810699:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8510 00002000:00100000:2.0:1769074770.810707:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1310720->1314815) req@ffff89b5461e6300 x1855007616820736/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/4536 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.810723:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e6300 with x1855007616820736 ext(1310720->1314815) 00002000:00010000:2.0:1769074770.810727:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5461e6300 00000100:00100000:2.0:1769074770.816397:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.816574:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.820400:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.822558:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616821120 00000100:00100000:0.0:1769074770.822573:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616821120 00002000:00100000:0.0:1769074770.822633:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54d101180 x1855007616821120/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 504/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.822650:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101180 with x1855007616821120 ext(1048576->2097151) 00002000:00010000:0.0:1769074770.822656:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d101180 00000100:00100000:0.0:1769074770.822701:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8511 00000100:00100000:0.0:1769074770.822711:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616821120:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074770.824771:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820992 00000100:00100000:1.0:1769074770.824785:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820992 00000100:00100000:0.0:1769074770.830360:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.830443:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616821376 00000100:00100000:2.0:1769074770.830454:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616821376 00000100:00100000:2.0:1769074770.830515:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13314 00000100:00100000:2.0:1769074770.830525:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:62825:x1855007616821376:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074770.830550:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.830570:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879baf6 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.830649:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830664:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830675:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830690:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830697:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830703:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830846:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538e8 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830858:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830865:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538e8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074770.830875:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074770.830958:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830966:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830971:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830978:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830983:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea5400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074770.830987:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.830993:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831005:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074770.831008:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074770.831013:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831023:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd538da lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831035:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831040:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831056:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831062:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074770.831064:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831070:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831077:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4000) 00010000:00010000:2.0:1769074770.831079:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074770.831082:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831087:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd538e1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074770.831126:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd538d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baf6 expref: 242 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.831138:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538e8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baf6 expref: 241 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.831150:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074770.831153:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074770.831155:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4e00, rc 0) 00000100:00100000:2.0:1769074770.831216:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62825:x1855007616821376:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 692us (879us total) trans 8589943935 rc 0/0 00000100:00100000:2.0:1769074770.831224:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13314 00000100:00100000:1.0:1769074770.831555:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616820864 00000100:00100000:1.0:1769074770.831562:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616820864 00000100:00100000:1.0:1769074770.831600:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14022 00000100:00100000:1.0:1769074770.831609:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616820992:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.831620:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.831631:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd538f6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879baef expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074770.831676:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625130752:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074770.832651:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.832681:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14023 00000100:00100000:1.0:1769074770.832700:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616820864:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074770.832711:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074770.832724:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bae8 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074770.832742:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074770.832744:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074770.832753:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 33 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.832773:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 33 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.832785:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:1.0:1769074770.832824:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616820864:12345-192.168.204.60@tcp:101:cat.0 Request processed in 128us (16250us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074770.832832:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14023 00002000:00010000:1.0:1769074770.838030:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074769 -> 1769074770 00002000:00010000:1.0:1769074770.838043:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074769 -> 1769074770 00002000:00010000:1.0:1769074770.838045:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074769 -> 1769074770 00000100:00100000:1.0:1769074770.838069:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625130752:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074770.838118:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd538f6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baef expref: 28 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.838142:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd538f6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baef expref: 28 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074770.838168:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:1.0:1769074770.838238:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616820992:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6628us (21852us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074770.838247:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14022 00000100:00100000:0.0:1769074770.840691:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616821120:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 17970us (20289us total) trans 0 rc 1044480/1044480 00000100:00100000:0.0:1769074770.840705:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8511 00002000:00100000:0.0:1769074770.840717:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54d101180 x1855007616821120/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 504/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 1044480/1044480 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.840726:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101180 with x1855007616821120 ext(1048576->2097151) 00002000:00010000:0.0:1769074770.840732:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d101180 00000100:00100000:0.0:1769074770.846588:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.846648:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.846743:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616821760 00000100:00100000:3.0:1769074770.846757:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616821760 00000100:00100000:3.0:1769074770.846958:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1004 00000100:00100000:3.0:1769074770.846968:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:2408:x1855007616821760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.846987:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.847016:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd538e8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baf6 expref: 241 pid: 6513 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074770.847210:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616821632 00000100:00100000:2.0:1769074770.847216:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616821632 00000100:00100000:2.0:1769074770.847252:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4662 00000100:00100000:2.0:1769074770.847260:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62825:x1855007616821632:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074770.847629:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:62825:x1855007616821632:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 367us (1055us total) trans 8589943936 rc 0/0 00000100:00100000:2.0:1769074770.847639:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4662 00010000:00010000:3.0:1769074770.847688:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074770.847695:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.847718:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:2408:x1855007616821760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 751us (1069us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.847726:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1004 00000100:00100000:1.0:1769074770.854401:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.854528:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616821888 00000100:00100000:0.0:1769074770.854541:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616821888 00002000:00100000:0.0:1769074770.854601:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5437b4e00 x1855007616821888/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.854613:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4e00 with x1855007616821888 ext(0->1048575) 00002000:00010000:0.0:1769074770.854618:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4e00 00000100:00100000:0.0:1769074770.854656:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8512 00000100:00100000:0.0:1769074770.854666:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2407:x1855007616821888:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074770.876539:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.876909:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074770.877159:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2407:x1855007616821888:12345-192.168.204.60@tcp:3:cat.0 Request processed in 22491us (22769us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074770.877173:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8512 00002000:00100000:2.0:1769074770.877189:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5437b4e00 x1855007616821888/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/440 e 0 to 0 dl 1769074781 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.877199:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4e00 with x1855007616821888 ext(0->1048575) 00002000:00010000:2.0:1769074770.877207:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4e00 00000100:00100000:1.0:1769074770.890006:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822144 00000100:00100000:1.0:1769074770.890029:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822144 00000100:00100000:1.0:1769074770.890374:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822016 00000100:00100000:1.0:1769074770.890379:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822016 00000100:00100000:2.0:1769074770.890471:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14024 00000100:00100000:2.0:1769074770.890485:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62820:x1855007616822144:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074770.890505:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.890525:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb04 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.890557:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.890568:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.890570:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074770.890576:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.890620:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.890637:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 32 pid: 8396 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.890653:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625130880:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.890783:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.895091:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14025 00000100:00100000:2.0:1769074770.895378:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62814:x1855007616822016:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074770.895402:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074770.895429:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bafd expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.895466:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.895481:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 26 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.895489:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.895492:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.895496:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1114111] (req 1048576->1114111) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.895534:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 26 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.895548:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 26 pid: 28954 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.895562:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625131008:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074770.895674:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.895683:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.895692:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625131136:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074770.895716:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074770.911469:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.911547:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625131008:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074770.911561:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625131136:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074770.911598:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1114111] (req 1048576->1114111) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074770.911619:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45600, rc 0) 00000100:00100000:3.0:1769074770.911681:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62814:x1855007616822016:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 16576us (34773us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.911692:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14025 00000100:00100000:3.0:1769074770.911742:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822272 00000100:00100000:3.0:1769074770.911751:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822272 00002000:00100000:3.0:1769074770.911801:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1298432->1605631) req@ffff89b54510a680 x1855007616822272/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074770.911918:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510a680 with x1855007616822272 ext(1298432->1605631) 00010000:00010000:3.0:1769074770.911927:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 33 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.911939:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 33 pid: 8396 timeout: 1858 lvb_type: 0 00002000:00010000:3.0:1769074770.911946:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54510a680 00000100:00100000:3.0:1769074770.911969:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 752 00000100:00100000:3.0:1769074770.911977:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616822272:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074770.913135:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625130880:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074770.913170:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.913186:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00000100:00100000:0.0:1769074770.913227:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62820:x1855007616822144:12345-192.168.204.60@tcp:101:cat.0 Request processed in 22746us (36705us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.913236:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14024 00000100:00100000:2.0:1769074770.930947:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.931108:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822400 00000100:00100000:3.0:1769074770.931123:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822400 00010000:00010000:3.0:1769074770.931185:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074770.931279:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6994 00000100:00100000:3.0:1769074770.931296:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2406:x1855007616822400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074770.931317:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074770.931356:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.931370:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.931399:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd537bb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba24 expref: 23 pid: 8398 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074770.931420:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.931492:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616822400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 199us (557us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.931502:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6994 00000100:00100000:3.0:1769074770.934664:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074770.934732:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.936637:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822528 00000100:00100000:0.0:1769074770.936656:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822528 00000100:00100000:0.0:1769074770.936752:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14026 00000100:00100000:0.0:1769074770.936769:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62823:x1855007616822528:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074770.936792:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074770.936882:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb0b expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.936930:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.936940:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074770.936943:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074770.936948:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.937003:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.937026:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 21 pid: 28954 timeout: 1858 lvb_type: 0 00000100:00100000:0.0:1769074770.937067:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625131264:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074770.937241:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b68160ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074770.943444:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822656 00000100:00100000:1.0:1769074770.943464:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822656 00010000:00010000:2.0:1769074770.943663:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 29 pid: 28954 timeout: 1858 lvb_type: 0 00000100:00100000:2.0:1769074770.943717:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6995 00000100:00100000:2.0:1769074770.943728:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616822656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074770.943746:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074770.943771:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 29 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.943786:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 29 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.943854:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd537de lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba47 expref: 29 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.943873:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.943876:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074770.943885:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.943905:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074770.943974:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074770.944001:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625131392:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074770.944237:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074770.949117:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625131264:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074770.949174:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 28 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074770.949202:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866600, rc 0) 00000100:00100000:0.0:1769074770.949303:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62823:x1855007616822528:12345-192.168.204.60@tcp:101:cat.0 Request processed in 12536us (14647us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074770.949316:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14026 00000100:00100000:0.0:1769074770.963556:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074770.963606:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822784 00000100:00100000:0.0:1769074770.963618:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822784 00002000:00100000:0.0:1769074770.963672:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1310720->2097151) req@ffff89b54d103100 x1855007616822784/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/0 e 0 to 0 dl 1769074781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074770.963684:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103100 with x1855007616822784 ext(1310720->2097151) 00010000:00010000:0.0:1769074770.963694:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 22 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074770.963705:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 22 pid: 28954 timeout: 1858 lvb_type: 0 00002000:00010000:0.0:1769074770.963718:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d103100 00000100:00100000:0.0:1769074770.963746:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 753 00000100:00100000:0.0:1769074770.963756:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616822784:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074770.975740:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2408:x1855007616822272:12345-192.168.204.60@tcp:4:cat.0 Request processed in 63746us (64265us total) trans 17179874506 rc 0/0 00000100:00100000:2.0:1769074770.975761:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 752 00002000:00100000:2.0:1769074770.975778:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2be:0x0] (1298432->1605631) req@ffff89b54510a680 x1855007616822272/t17179874506(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:306/0 lens 488/448 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074770.975791:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510a680 with x1855007616822272 ext(1298432->1605631) 00000100:00100000:3.0:1769074770.979485:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625131392:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074770.979522:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074770.979732:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616822656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 36002us (44996us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074770.979748:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6995 00010000:00010000:2.0:1769074770.979963:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 35 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074770.979994:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 35 pid: 8396 timeout: 1858 lvb_type: 0 00002000:00010000:2.0:1769074770.980006:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54510a680 00000100:00100000:0.0:1769074771.019224:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.019504:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616822912 00000100:00100000:2.0:1769074771.019516:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616822912 00002000:00100000:2.0:1769074771.019590:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1110016->1114111) req@ffff89b54ce9c380 x1855007616822912/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.019606:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9c380 with x1855007616822912 ext(1110016->1114111) 00002000:00010000:2.0:1769074771.019615:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9c380 00000100:00100000:2.0:1769074771.019674:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8513 00000100:00100000:2.0:1769074771.019688:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2407:x1855007616822912:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074771.019975:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2407:x1855007616822912:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 287us (762us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074771.019985:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8513 00002000:00100000:2.0:1769074771.019995:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1110016->1114111) req@ffff89b54ce9c380 x1855007616822912/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.020006:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9c380 with x1855007616822912 ext(1110016->1114111) 00002000:00010000:2.0:1769074771.020010:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9c380 00000100:00080000:1.0:1769074771.024534:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074771.046110:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.047583:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2409:x1855007616822784:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 83813us (84023us total) trans 21474841191 rc 0/0 00000100:00100000:2.0:1769074771.047604:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 753 00002000:00100000:2.0:1769074771.047624:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1310720->2097151) req@ffff89b54d103100 x1855007616822784/t21474841191(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:306/0 lens 488/448 e 0 to 0 dl 1769074781 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.047635:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d103100 with x1855007616822784 ext(1310720->2097151) 00010000:00010000:2.0:1769074771.047648:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 24 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:2.0:1769074771.047666:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 24 pid: 28954 timeout: 1858 lvb_type: 0 00002000:00010000:2.0:1769074771.047673:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d103100 00000100:00100000:3.0:1769074771.051644:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823040 00000100:00100000:3.0:1769074771.051665:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823040 00010000:00010000:0.0:1769074771.053475:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 35 pid: 8396 timeout: 1858 lvb_type: 0 00000100:00100000:0.0:1769074771.053554:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6996 00000100:00100000:0.0:1769074771.053571:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2407:x1855007616823040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.053591:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.053617:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 35 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.053633:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 35 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.053662:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ee00/0x7fe2323adfd5388d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1298432->1605631) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879babe expref: 35 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.053684:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.053687:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.053696:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.053707:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.053758:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074771.053776:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625131520:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074771.054175:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b68160ae40 going to sleep for 11 seconds 00000100:00080000:1.0:1769074771.055026:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:0.0:1769074771.056733:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841191 is committed 00000001:00080000:0.0:1769074771.056751:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841190 is committed 00000100:00100000:2.0:1769074771.059674:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.059767:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823168 00000100:00100000:3.0:1769074771.059777:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823168 00010000:00010000:3.0:1769074771.059950:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 10 pid: 28954 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074771.059996:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6997 00000100:00100000:3.0:1769074771.060005:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616823168:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.060021:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074771.060045:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 10 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074771.060064:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 10 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074771.060096:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b08e800/0x7fe2323adfd538a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1310720->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bada expref: 10 pid: 28954 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074771.060121:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.060124:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.060135:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 14 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.060152:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 14 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.060206:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 14 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.060225:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625131648:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074771.060346:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074771.064540:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625131520:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074771.064577:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.064655:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2407:x1855007616823040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11090us (18560us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.064667:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6996 00000100:00100000:1.0:1769074771.082708:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.082965:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823296 00000100:00100000:2.0:1769074771.082976:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823296 00002000:00100000:2.0:1769074771.083046:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5437b4a80 x1855007616823296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.083057:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4a80 with x1855007616823296 ext(1048576->2097151) 00002000:00010000:2.0:1769074771.083064:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4a80 00000100:00080000:1.0:1769074771.083408:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074771.084083:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8514 00000100:00100000:2.0:1769074771.084109:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616823296:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074771.095383:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625131648:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074771.095415:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074771.103775:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616823168:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 43754us (44095us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074771.103793:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6997 00000100:00100000:2.0:1769074771.104531:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.104764:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823424 00000100:00100000:3.0:1769074771.104776:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823424 00002000:00100000:3.0:1769074771.105082:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54d0e7100 x1855007616823424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.105096:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e7100 with x1855007616823424 ext(1048576->2097151) 00002000:00010000:3.0:1769074771.105104:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e7100 00000100:00100000:3.0:1769074771.105137:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8515 00000100:00100000:3.0:1769074771.105149:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616823424:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074771.118352:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616823296:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 34232us (35632us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074771.118370:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8514 00002000:00100000:0.0:1769074771.118387:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5437b4a80 x1855007616823296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.118399:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4a80 with x1855007616823296 ext(1048576->2097151) 00002000:00010000:0.0:1769074771.118407:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4a80 00000100:00100000:3.0:1769074771.129878:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616823424:12345-192.168.204.60@tcp:3:cat.0 Request processed in 24660us (25281us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074771.129896:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8515 00002000:00100000:3.0:1769074771.129915:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54d0e7100 x1855007616823424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.129933:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d0e7100 with x1855007616823424 ext(1048576->2097151) 00002000:00010000:3.0:1769074771.129941:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d0e7100 00000100:00100000:1.0:1769074771.140523:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.140636:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823552 00000100:00100000:0.0:1769074771.140647:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823552 00000100:00100000:0.0:1769074771.140721:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14027 00000100:00100000:0.0:1769074771.140731:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:62823:x1855007616823552:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074771.140752:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074771.140772:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb12 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.140848:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.140858:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074771.140860:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074771.140865:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.140901:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.140913:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 8 pid: 8396 timeout: 1858 lvb_type: 0 00000100:00100000:0.0:1769074771.140927:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625131776:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074771.140998:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b68160ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.147309:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.147419:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823808 00000100:00100000:3.0:1769074771.147431:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823808 00002000:00100000:3.0:1769074771.147490:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b546f91c00 x1855007616823808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.147502:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91c00 with x1855007616823808 ext(2097152->4194303) 00002000:00010000:3.0:1769074771.147509:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f91c00 00000100:00100000:3.0:1769074771.147546:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8516 00000100:00100000:3.0:1769074771.147556:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616823808:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.149443:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.149561:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823680 00000100:00100000:3.0:1769074771.149570:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823680 00000100:00100000:3.0:1769074771.149613:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14028 00000100:00100000:3.0:1769074771.149622:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62825:x1855007616823680:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074771.149640:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074771.149658:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb19 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.149689:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 8 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.149699:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.149711:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.149713:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.149717:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2527232->3145727] (req 2527232->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.149752:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.149763:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 1858 lvb_type: 1 00000100:00100000:3.0:1769074771.149777:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625131904:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074771.149883:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 8 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.149892:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 8 pid: 9048 timeout: 1858 lvb_type: 0 00000100:00100000:3.0:1769074771.149901:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625132032:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074771.149924:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:1.0:1769074771.150077:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.150173:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625131776:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074771.150197:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.150211:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866000, rc 0) 00000100:00100000:0.0:1769074771.150254:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62823:x1855007616823552:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9524us (9742us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.150262:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14027 00000100:00100000:3.0:1769074771.150404:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616823936 00000100:00100000:3.0:1769074771.150409:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616823936 00002000:00100000:3.0:1769074771.150441:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1114111) req@ffff89b5437b7480 x1855007616823936/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.150451:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b7480 with x1855007616823936 ext(1048576->1114111) 00010000:00010000:3.0:1769074771.150459:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 9 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:3.0:1769074771.150470:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 9 pid: 8396 timeout: 1858 lvb_type: 0 00002000:00010000:3.0:1769074771.150477:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b7480 00000100:00100000:3.0:1769074771.150540:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 754 00000100:00100000:3.0:1769074771.150548:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616823936:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074771.155675:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625131904:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074771.161328:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.161436:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824064 00000100:00100000:0.0:1769074771.161448:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824064 00010000:00010000:0.0:1769074771.161493:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 1858 lvb_type: 1 00000100:00100000:0.0:1769074771.161541:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6998 00000100:00100000:0.0:1769074771.161554:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b653a99850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616824064:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.161572:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.161597:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 1858 lvb_type: 1 00010000:00010000:0.0:1769074771.161612:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 1858 lvb_type: 1 00010000:00010000:0.0:1769074771.161643:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd53840 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba8d expref: 18 pid: 8398 timeout: 1858 lvb_type: 1 00010000:00010000:0.0:1769074771.161664:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.161736:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b653a99850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616824064:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 185us (420us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.161743:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6998 00000100:00100000:3.0:1769074771.163340:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625132032:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074771.163386:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2527232->3145727] (req 2527232->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.163406:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9000, rc 0) 00000100:00100000:3.0:1769074771.165197:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:62825:x1855007616823680:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 15565us (15757us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074771.165213:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14028 00000100:00100000:0.0:1769074771.165531:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616823936:12345-192.168.204.60@tcp:4:cat.0 Request processed in 14981us (15451us total) trans 21474841192 rc 0/0 00000100:00100000:0.0:1769074771.165542:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 754 00002000:00100000:0.0:1769074771.165554:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1114111) req@ffff89b5437b7480 x1855007616823936/t21474841192(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/448 e 0 to 0 dl 1769074782 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.165566:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b7480 with x1855007616823936 ext(1048576->1114111) 00010000:00010000:0.0:1769074771.165576:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 11 pid: 8396 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.165589:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 11 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:0.0:1769074771.165598:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b7480 00000100:00100000:1.0:1769074771.165671:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.166033:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616823808:12345-192.168.204.60@tcp:3:cat.0 Request processed in 18477us (18737us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074771.166042:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8516 00002000:00100000:2.0:1769074771.166050:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b546f91c00 x1855007616823808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.166059:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91c00 with x1855007616823808 ext(2097152->4194303) 00002000:00010000:2.0:1769074771.166062:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f91c00 00000100:00100000:0.0:1769074771.166490:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824192 00000100:00100000:0.0:1769074771.166499:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824192 00010000:00010000:0.0:1769074771.166529:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 11 pid: 9048 timeout: 1858 lvb_type: 0 00000100:00100000:0.0:1769074771.166625:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 6999 00000100:00100000:0.0:1769074771.166634:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616824192:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.166648:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.166671:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 11 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.166684:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 11 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.166708:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd537e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba4e expref: 11 pid: 9048 timeout: 1858 lvb_type: 0 00010000:00010000:0.0:1769074771.166729:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.166731:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.166738:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.166747:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.166783:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074771.166797:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625132160:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074771.166887:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074771.173420:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.173580:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.173726:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824448 00000100:00100000:3.0:1769074771.173737:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824448 00000100:00100000:3.0:1769074771.173802:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14029 00000100:00100000:3.0:1769074771.173833:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62820:x1855007616824448:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074771.173851:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074771.173871:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb20 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.173907:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.173915:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074771.173917:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074771.173921:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 12 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.173963:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.173976:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 15 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.173993:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625132288:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074771.174083:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074771.174531:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625132160:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074771.174549:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.174590:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616824192:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7957us (8920us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.174599:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 6999 00000100:00100000:3.0:1769074771.174708:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.174941:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824576 00000100:00100000:0.0:1769074771.174948:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824576 00010000:00010000:0.0:1769074771.174971:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 12 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:0.0:1769074771.175000:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7000 00000100:00100000:0.0:1769074771.175007:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616824576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.175018:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1769074771.175040:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074769 -> 1769074770 00002000:00010000:0.0:1769074771.175043:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074769 -> 1769074770 00010000:00010000:0.0:1769074771.175051:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 12 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.175064:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 12 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.175090:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd5390b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1114111) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bafd expref: 12 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.175111:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.175113:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.175123:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.175134:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.175167:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074771.175180:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625132416:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074771.175217:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824320 00000100:00100000:2.0:1769074771.175223:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824320 00000100:00100000:0.0:1769074771.175238:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00002000:00100000:2.0:1769074771.175261:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2527232->2531327) req@ffff89b5437b5500 x1855007616824320/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.175271:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5500 with x1855007616824320 ext(2527232->2531327) 00010000:00010000:2.0:1769074771.175278:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074771.175290:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:2.0:1769074771.175298:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b5500 00000100:00100000:1.0:1769074771.176437:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625132288:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074771.176474:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 11 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.176490:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8e00, rc 0) 00000100:00100000:1.0:1769074771.176537:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62820:x1855007616824448:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2705us (2957us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.176545:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14029 00000100:00100000:2.0:1769074771.176558:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 755 00000100:00100000:2.0:1769074771.176570:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616824320:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074771.176701:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616824320:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 134us (3292us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074771.176708:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 755 00002000:00100000:2.0:1769074771.176716:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2527232->2531327) req@ffff89b5437b5500 x1855007616824320/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.176725:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5500 with x1855007616824320 ext(2527232->2531327) 00010000:00010000:2.0:1769074771.176732:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074771.176744:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:2.0:1769074771.176751:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b5500 00000100:00100000:0.0:1769074771.177593:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625132416:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074771.177616:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.178050:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616824576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3041us (3340us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.178061:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7000 00000100:00100000:1.0:1769074771.181380:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.181523:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824704 00000100:00100000:0.0:1769074771.181534:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824704 00002000:00100000:0.0:1769074771.181584:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5437b7100 x1855007616824704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.181597:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b7100 with x1855007616824704 ext(1048576->2097151) 00002000:00010000:0.0:1769074771.181603:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5437b7100 00000100:00100000:0.0:1769074771.181690:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8517 00000100:00100000:0.0:1769074771.181701:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616824704:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074771.192560:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616824704:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 10846us (11176us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074771.192577:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8517 00002000:00100000:1.0:1769074771.192594:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5437b7100 x1855007616824704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.192604:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b7100 with x1855007616824704 ext(1048576->2097151) 00002000:00010000:1.0:1769074771.192610:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5437b7100 00000100:00100000:3.0:1769074771.196578:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.196757:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.196952:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824832 00000100:00100000:2.0:1769074771.196963:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824832 00002000:00100000:2.0:1769074771.197017:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2527232->3145727) req@ffff89b545108000 x1855007616824832/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.197030:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108000 with x1855007616824832 ext(2527232->3145727) 00010000:00010000:2.0:1769074771.197040:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074771.197056:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 16 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:2.0:1769074771.197065:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545108000 00000100:00100000:2.0:1769074771.197103:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 756 00000100:00100000:2.0:1769074771.197112:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2406:x1855007616824832:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074771.198434:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616824960 00000100:00100000:1.0:1769074771.198445:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616824960 00000100:00100000:1.0:1769074771.198532:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14030 00000100:00100000:1.0:1769074771.198542:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:62825:x1855007616824960:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074771.198555:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074771.198574:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb27 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.198597:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.198607:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.198614:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074771.198616:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074771.198620:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3547135] (req 3145728->3547135) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.198657:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.198669:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 20 pid: 9048 timeout: 1859 lvb_type: 0 00000100:00100000:1.0:1769074771.198695:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625132544:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074771.198826:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.198837:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 35 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:1.0:1769074771.198848:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625132672:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074771.198879:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074771.203072:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.204434:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825088 00000100:00100000:0.0:1769074771.204454:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825088 00000100:00100000:0.0:1769074771.204578:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14031 00000100:00100000:0.0:1769074771.204588:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:62823:x1855007616825088:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074771.204607:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074771.204627:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb2e expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.204654:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.204657:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.204665:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.204682:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.204696:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866200, rc 0) 00000100:00100000:0.0:1769074771.204766:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:62823:x1855007616825088:12345-192.168.204.60@tcp:101:cat.0 Request processed in 179us (1709us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.204773:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14031 00000100:00100000:1.0:1769074771.208255:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625132544:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074771.208275:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625132672:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074771.208314:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3547135] (req 3145728->3547135) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 36 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.208349:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:1.0:1769074771.208468:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:62825:x1855007616824960:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 9925us (11898us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.208479:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14030 00000100:00100000:1.0:1769074771.214007:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2406:x1855007616824832:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 16885us (17239us total) trans 21474841193 rc 0/0 00000100:00100000:1.0:1769074771.214022:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 756 00002000:00100000:1.0:1769074771.214034:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2527232->3145727) req@ffff89b545108000 x1855007616824832/t21474841193(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/448 e 0 to 0 dl 1769074782 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.214050:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108000 with x1855007616824832 ext(2527232->3145727) 00010000:00010000:1.0:1769074771.214060:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.214076:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:1.0:1769074771.214083:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545108000 00000100:00100000:2.0:1769074771.219621:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.219665:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.219763:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.219843:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825216 00000100:00100000:1.0:1769074771.219853:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825216 00002000:00100000:1.0:1769074771.219903:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546f91180 x1855007616825216/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.219913:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91180 with x1855007616825216 ext(1048576->2097151) 00002000:00010000:1.0:1769074771.219919:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f91180 00000100:00100000:1.0:1769074771.219950:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8518 00000100:00100000:1.0:1769074771.219959:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2407:x1855007616825216:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074771.222550:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825472 00000100:00100000:0.0:1769074771.222564:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825472 00010000:00010000:0.0:1769074771.222610:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 21 pid: 9048 timeout: 1859 lvb_type: 0 00000100:00100000:0.0:1769074771.222656:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825344 00000100:00100000:0.0:1769074771.222660:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825344 00010000:00010000:0.0:1769074771.222673:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 38 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:0.0:1769074771.222697:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7001 00000100:00100000:0.0:1769074771.222705:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616825472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.222721:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.222743:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 21 pid: 9048 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.222757:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 21 pid: 9048 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.222785:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd5385c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 4804608->5242879) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879baa9 expref: 21 pid: 9048 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.222842:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.222899:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616825472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 194us (3232us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.222908:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7001 00000100:00100000:0.0:1769074771.222928:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7002 00000100:00100000:0.0:1769074771.222934:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2408:x1855007616825344:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.222943:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.222954:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 38 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.222965:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 38 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.222978:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67602ae00/0x7fe2323adfd53847 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ba94 expref: 38 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.222988:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.222990:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.222996:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.223006:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.223053:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074771.223072:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625132800:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074771.223131:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.230646:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.232534:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625132800:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074771.232574:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.232669:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616825344:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9732us (12903us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.232682:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7002 00000100:00100000:0.0:1769074771.232725:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825600 00000100:00100000:0.0:1769074771.232732:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825600 00010000:00010000:0.0:1769074771.232773:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:1.0:1769074771.234394:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7003 00000100:00100000:1.0:1769074771.234427:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616825600:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074771.234462:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074771.234491:0:6499:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074769 -> 1769074770 00002000:00010000:1.0:1769074771.234495:0:6499:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074769 -> 1769074770 00010000:00010000:1.0:1769074771.234504:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.234532:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.234563:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9000/0x7fe2323adfd53920 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2527232->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb19 expref: 18 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.234583:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074771.234585:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074771.234596:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.234612:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.234680:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074771.234695:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625132928:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074771.234828:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.235000:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2407:x1855007616825216:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 15039us (15387us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074771.235009:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8518 00002000:00100000:2.0:1769074771.235018:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546f91180 x1855007616825216/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.235029:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f91180 with x1855007616825216 ext(1048576->2097151) 00002000:00010000:2.0:1769074771.235033:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f91180 00000100:00100000:3.0:1769074771.235351:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.235537:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825728 00000100:00100000:1.0:1769074771.235547:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825728 00002000:00100000:1.0:1769074771.235581:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3543040->3547135) req@ffff89b545109500 x1855007616825728/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.235591:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109500 with x1855007616825728 ext(3543040->3547135) 00002000:00010000:1.0:1769074771.235596:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545109500 00000100:00100000:1.0:1769074771.235627:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8519 00000100:00100000:1.0:1769074771.235634:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616825728:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074771.235793:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616825728:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 159us (442us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074771.235800:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8519 00002000:00100000:1.0:1769074771.235839:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3543040->3547135) req@ffff89b545109500 x1855007616825728/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.235847:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109500 with x1855007616825728 ext(3543040->3547135) 00002000:00010000:1.0:1769074771.235851:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545109500 00000100:00100000:1.0:1769074771.240893:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625132928:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074771.240924:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074771.240977:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616825600:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6568us (10341us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.240987:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7003 00000100:00100000:3.0:1769074771.245719:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.245843:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825856 00000100:00100000:2.0:1769074771.245855:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825856 00002000:00100000:2.0:1769074771.245912:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b545109f80 x1855007616825856/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.245925:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109f80 with x1855007616825856 ext(2097152->3145727) 00002000:00010000:2.0:1769074771.245931:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545109f80 00000100:00100000:2.0:1769074771.245974:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8520 00000100:00100000:2.0:1769074771.245984:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616825856:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074771.257412:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616825856:12345-192.168.204.60@tcp:3:cat.0 Request processed in 11414us (11687us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074771.257429:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8520 00002000:00100000:2.0:1769074771.257443:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b545109f80 x1855007616825856/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.257460:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109f80 with x1855007616825856 ext(2097152->3145727) 00002000:00010000:2.0:1769074771.257468:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545109f80 00000100:00100000:0.0:1769074771.269967:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.270142:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616825984 00000100:00100000:1.0:1769074771.270154:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616825984 00000100:00100000:1.0:1769074771.270217:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14032 00000100:00100000:1.0:1769074771.270227:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62820:x1855007616825984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074771.270247:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074771.270266:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb35 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.270301:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 34 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.270314:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074771.270316:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074771.270320:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.270327:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074771.270370:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 34 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.270384:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 34 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:1.0:1769074771.270403:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625133056:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074771.270415:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826112 00000100:00100000:0.0:1769074771.270419:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826112 00002000:00100000:0.0:1769074771.270452:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b545108700 x1855007616826112/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.270461:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108700 with x1855007616826112 ext(2097152->3145727) 00010000:00010000:0.0:1769074771.270468:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 6/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 35 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.270479:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 6/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 35 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:0.0:1769074771.270486:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b545108700 00000100:00100000:1.0:1769074771.270500:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64d08ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074771.270502:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 757 00000100:00100000:0.0:1769074771.270509:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2407:x1855007616826112:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.281347:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625133056:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074771.281395:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.281421:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:1.0:1769074771.281497:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62820:x1855007616825984:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11270us (11541us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.281507:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14032 00000100:00100000:2.0:1769074771.283149:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.283244:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826240 00000100:00100000:3.0:1769074771.283257:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826240 00002000:00100000:3.0:1769074771.283318:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3547135) req@ffff89b54c845500 x1855007616826240/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.283339:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845500 with x1855007616826240 ext(3145728->3547135) 00010000:00010000:3.0:1769074771.283349:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.283365:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:3.0:1769074771.283373:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54c845500 00000100:00100000:3.0:1769074771.283415:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 758 00000100:00100000:3.0:1769074771.283426:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616826240:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074771.294287:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2407:x1855007616826112:12345-192.168.204.60@tcp:3:cat.0 Request processed in 23757us (23938us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074771.294305:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 757 00002000:00100000:1.0:1769074771.294324:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b545108700 x1855007616826112/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.294350:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108700 with x1855007616826112 ext(2097152->3145727) 00010000:00010000:1.0:1769074771.294362:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.294384:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:1.0:1769074771.294393:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b545108700 00000100:00100000:0.0:1769074771.298439:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616826240:12345-192.168.204.60@tcp:4:cat.0 Request processed in 14998us (15284us total) trans 17179874507 rc 0/0 00000100:00100000:0.0:1769074771.298456:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 758 00002000:00100000:0.0:1769074771.298471:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3547135) req@ffff89b54c845500 x1855007616826240/t17179874507(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/448 e 0 to 0 dl 1769074782 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.298483:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845500 with x1855007616826240 ext(3145728->3547135) 00010000:00010000:0.0:1769074771.298494:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074771.298515:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:0.0:1769074771.298524:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54c845500 00000100:00100000:3.0:1769074771.303513:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.303598:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826368 00000100:00100000:1.0:1769074771.303611:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826368 00000100:00100000:1.0:1769074771.303733:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14033 00000100:00100000:1.0:1769074771.303743:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67757a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62823:x1855007616826368:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074771.303761:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074771.303781:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb3c expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.303848:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074771.303850:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074771.303859:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.303875:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.303888:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:1.0:1769074771.303945:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67757a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62823:x1855007616826368:12345-192.168.204.60@tcp:101:cat.0 Request processed in 203us (446us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.303952:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14033 00000100:00100000:2.0:1769074771.308200:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.308373:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826496 00000100:00100000:3.0:1769074771.308388:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826496 00010000:00010000:3.0:1769074771.308455:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.308559:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7004 00000100:00100000:3.0:1769074771.308569:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2409:x1855007616826496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.308586:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1769074771.308606:0:6499:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074769 -> 1769074770 00002000:00010000:3.0:1769074771.308609:0:6499:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074769 -> 1769074770 00010000:00010000:3.0:1769074771.308616:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.308637:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.308670:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd5392e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3547135) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb27 expref: 37 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.308713:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.308717:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.308729:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.308744:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.308800:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.309133:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625133184:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074771.309232:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074771.314253:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625133184:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074771.314281:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074771.314360:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616826496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5792us (6171us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.314374:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7004 00000100:00100000:1.0:1769074771.320484:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.321151:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826624 00000100:00100000:0.0:1769074771.321164:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826624 00002000:00100000:0.0:1769074771.321226:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b5437b6d80 x1855007616826624/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.321238:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6d80 with x1855007616826624 ext(2097152->3145727) 00002000:00010000:0.0:1769074771.321246:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5437b6d80 00000100:00100000:0.0:1769074771.321305:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8521 00000100:00100000:0.0:1769074771.321320:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616826624:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074771.342087:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.342136:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.342380:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826752 00000100:00100000:0.0:1769074771.342395:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826752 00002000:00100000:0.0:1769074771.342460:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54c847480 x1855007616826752/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.342472:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c847480 with x1855007616826752 ext(3145728->4194303) 00002000:00010000:0.0:1769074771.342478:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c847480 00000100:00100000:0.0:1769074771.342512:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616826880 00000100:00100000:0.0:1769074771.342515:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616826880 00002000:00100000:0.0:1769074771.342530:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54c846680 x1855007616826880/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.342537:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c846680 with x1855007616826880 ext(3145728->4194303) 00002000:00010000:0.0:1769074771.342540:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c846680 00000100:00100000:0.0:1769074771.342562:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8522 00000100:00100000:0.0:1769074771.342571:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616826752:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074771.342574:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616826624:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21259us (22100us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074771.342582:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8521 00002000:00100000:3.0:1769074771.342588:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b5437b6d80 x1855007616826624/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.342595:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b6d80 with x1855007616826624 ext(2097152->3145727) 00002000:00010000:3.0:1769074771.342599:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5437b6d80 00000100:00100000:3.0:1769074771.342617:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8523 00000100:00100000:3.0:1769074771.342624:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616826880:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:0.0:1769074771.358030:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074771.377276:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616826880:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34634us (35120us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074771.377299:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8523 00002000:00100000:3.0:1769074771.377314:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54c846680 x1855007616826880/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074771.377325:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c846680 with x1855007616826880 ext(3145728->4194303) 00002000:00010000:3.0:1769074771.377341:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c846680 00000100:00100000:0.0:1769074771.380296:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616826752:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37714us (38213us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074771.380311:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8522 00002000:00100000:0.0:1769074771.380323:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b54c847480 x1855007616826752/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.380344:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c847480 with x1855007616826752 ext(3145728->4194303) 00002000:00010000:0.0:1769074771.380350:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c847480 00000100:00100000:1.0:1769074771.399150:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.399269:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827008 00000100:00100000:0.0:1769074771.399285:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827008 00000100:00100000:0.0:1769074771.399368:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14034 00000100:00100000:0.0:1769074771.399381:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:62823:x1855007616827008:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074771.399403:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074771.399429:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb43 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.399488:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.399492:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.399503:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.399522:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.399539:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:0.0:1769074771.399629:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:62823:x1855007616827008:12345-192.168.204.60@tcp:101:cat.0 Request processed in 250us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.399636:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14034 00000100:00100000:2.0:1769074771.405090:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.405152:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.405253:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827136 00000100:00100000:0.0:1769074771.405264:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827136 00000100:00100000:0.0:1769074771.405320:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827264 00000100:00100000:0.0:1769074771.405323:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827264 00000100:00100000:0.0:1769074771.405362:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14035 00000100:00100000:0.0:1769074771.405372:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616827136:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074771.405391:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074771.405409:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb4a expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.405438:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.405441:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.405451:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 12 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074771.405469:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 12 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074771.405481:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d800, rc 0) 00000100:00100000:0.0:1769074771.405537:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616827136:12345-192.168.204.60@tcp:101:cat.0 Request processed in 167us (456us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.405545:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14035 00000100:00100000:0.0:1769074771.405576:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14036 00000100:00100000:0.0:1769074771.405583:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616827264:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074771.405590:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074771.405597:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb51 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.405608:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.405610:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074771.405614:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 12 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074771.405624:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 12 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074771.405633:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:0.0:1769074771.405655:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616827264:12345-192.168.204.60@tcp:101:cat.0 Request processed in 73us (503us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.405664:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14036 00000100:00100000:3.0:1769074771.428986:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.429107:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827392 00000100:00100000:0.0:1769074771.429119:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827392 00002000:00100000:0.0:1769074771.429189:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b545109c00 x1855007616827392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.429201:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109c00 with x1855007616827392 ext(4194304->4808703) 00002000:00010000:0.0:1769074771.429207:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545109c00 00000100:00100000:0.0:1769074771.429245:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8524 00000100:00100000:0.0:1769074771.429254:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616827392:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.438953:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.439070:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.439995:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827520 00000100:00100000:0.0:1769074771.440011:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827520 00002000:00100000:0.0:1769074771.440086:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5437b4380 x1855007616827520/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.440099:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4380 with x1855007616827520 ext(3145728->4194303) 00002000:00010000:0.0:1769074771.440106:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4380 00000100:00100000:0.0:1769074771.440142:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827648 00000100:00100000:0.0:1769074771.440156:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827648 00002000:00100000:0.0:1769074771.440175:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b5437b4700 x1855007616827648/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.440183:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4700 with x1855007616827648 ext(3145728->4194303) 00002000:00010000:0.0:1769074771.440187:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5437b4700 00000100:00100000:0.0:1769074771.440223:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8525 00000100:00100000:0.0:1769074771.440231:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616827520:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.441380:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8526 00000100:00100000:1.0:1769074771.441396:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616827648:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.443986:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616827392:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14721us (14999us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074771.443999:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8524 00002000:00100000:1.0:1769074771.444012:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b545109c00 x1855007616827392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.444023:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109c00 with x1855007616827392 ext(4194304->4808703) 00002000:00010000:1.0:1769074771.444029:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545109c00 00000100:00100000:0.0:1769074771.464186:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.464631:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616827904 00000100:00100000:3.0:1769074771.464648:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616827904 00000100:00100000:3.0:1769074771.464737:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13315 00000100:00100000:3.0:1769074771.464750:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62831:x1855007616827904:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074771.464786:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074771.464847:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd5395f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb58 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.464932:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.464947:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.464958:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.464978:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.464985:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.464991:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465147:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd53974 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465162:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd53974 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465170:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd53974 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074771.465183:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074771.465292:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465300:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465307:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465313:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465321:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074771.465325:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465345:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465358:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074771.465361:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074771.465368:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465375:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53966 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465392:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465398:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465404:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465410:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074771.465413:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465420:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465427:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6543ea900) 00010000:00010000:3.0:1769074771.465429:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074771.465432:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465437:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd5396d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074771.465485:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9600/0x7fe2323adfd5395f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb58 expref: 244 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.465499:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd53974 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb58 expref: 243 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.465506:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074771.465508:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074771.465511:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8a00, rc 0) 00000100:00100000:3.0:1769074771.465567:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62831:x1855007616827904:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 820us (1397us total) trans 8589943937 rc 0/0 00000100:00100000:3.0:1769074771.465574:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13315 00000100:00100000:1.0:1769074771.472152:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616827648:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30741us (33061us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074771.472172:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8526 00002000:00100000:1.0:1769074771.472193:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b5437b4700 x1855007616827648/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.472206:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4700 with x1855007616827648 ext(3145728->4194303) 00002000:00010000:1.0:1769074771.472213:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5437b4700 00000100:00100000:3.0:1769074771.477005:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.477077:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.477136:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828160 00000100:00100000:2.0:1769074771.477149:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828160 00000100:00100000:2.0:1769074771.477218:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4663 00000100:00100000:2.0:1769074771.477229:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62831:x1855007616828160:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074771.477508:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:62831:x1855007616828160:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 280us (517us total) trans 8589943938 rc 0/0 00000100:00100000:2.0:1769074771.477516:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4663 00000100:00100000:1.0:1769074771.477774:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828288 00000100:00100000:1.0:1769074771.477780:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828288 00000100:00100000:1.0:1769074771.477853:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1005 00000100:00100000:1.0:1769074771.477861:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b653a99450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:2408:x1855007616828288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074771.477872:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074771.477896:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd53974 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb58 expref: 244 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074771.477915:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074771.477920:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074771.477946:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b653a99450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:2408:x1855007616828288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 87us (869us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.477955:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1005 00000100:00100000:0.0:1769074771.478335:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616827520:12345-192.168.204.60@tcp:3:cat.0 Request processed in 38093us (39388us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074771.478344:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8525 00002000:00100000:0.0:1769074771.478355:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5437b4380 x1855007616827520/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.478365:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b4380 with x1855007616827520 ext(3145728->4194303) 00002000:00010000:0.0:1769074771.478369:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b4380 00000100:00100000:3.0:1769074771.507241:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.507407:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828416 00000100:00100000:1.0:1769074771.507420:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828416 00002000:00100000:1.0:1769074771.507475:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54510b800 x1855007616828416/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.507488:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510b800 with x1855007616828416 ext(4194304->4808703) 00002000:00010000:1.0:1769074771.507494:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54510b800 00000100:00100000:1.0:1769074771.507616:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8527 00000100:00100000:1.0:1769074771.507630:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616828416:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.525765:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616828416:12345-192.168.204.60@tcp:3:cat.0 Request processed in 18118us (18515us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074771.525786:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8527 00002000:00100000:1.0:1769074771.525859:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54510b800 x1855007616828416/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.525873:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510b800 with x1855007616828416 ext(4194304->4808703) 00002000:00010000:1.0:1769074771.525883:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54510b800 00000100:00100000:1.0:1769074771.563654:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.564104:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828544 00000100:00100000:2.0:1769074771.564116:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828544 00000100:00100000:2.0:1769074771.564185:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14037 00000100:00100000:2.0:1769074771.564195:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62829:x1855007616828544:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074771.564216:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074771.564238:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb5f expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.564273:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.564298:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.564305:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.564307:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.564311:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2498560->3145727] (req 2498560->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.564358:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.564372:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 17 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.564388:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625133312:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074771.564489:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.564499:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.564509:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625133440:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074771.574236:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.574278:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625133312:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074771.575576:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.576168:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828672 00000100:00100000:1.0:1769074771.576179:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828672 00010000:00010000:1.0:1769074771.576224:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:1.0:1769074771.576270:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7005 00000100:00100000:1.0:1769074771.576280:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616828672:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074771.576305:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074771.576336:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.576349:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.576370:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53943 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb3c expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074771.576393:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074771.576452:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616828672:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 173us (884us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074771.576459:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7005 00000100:00100000:2.0:1769074771.583028:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625133440:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074771.583092:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2498560->3145727] (req 2498560->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.583118:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b4a00, rc 0) 00000100:00100000:2.0:1769074771.583203:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62829:x1855007616828544:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 19009us (19557us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074771.583215:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14037 00000100:00100000:2.0:1769074771.592412:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.592515:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828800 00000100:00100000:3.0:1769074771.592528:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828800 00010000:00010000:3.0:1769074771.592583:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.592630:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7006 00000100:00100000:3.0:1769074771.592641:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616828800:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.592661:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074771.592688:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.592710:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.592739:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53927 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb20 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.592758:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.592760:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.592769:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 12 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.592781:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 12 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.592893:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 12 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.592912:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625133568:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074771.593014:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:3.0:1769074771.597268:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.597487:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616828928 00000100:00100000:1.0:1769074771.597498:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616828928 00002000:00100000:1.0:1769074771.597552:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2498560->2502655) req@ffff89b545108380 x1855007616828928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.597562:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108380 with x1855007616828928 ext(2498560->2502655) 00002000:00010000:1.0:1769074771.597568:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545108380 00000100:00100000:0.0:1769074771.597716:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625133568:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074771.597740:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074771.597762:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8528 00000100:00100000:1.0:1769074771.597775:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616828928:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074771.597829:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616828800:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5188us (5426us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.597841:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7006 00000100:00100000:1.0:1769074771.598041:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616828928:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 268us (786us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074771.598049:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8528 00002000:00100000:1.0:1769074771.598056:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2498560->2502655) req@ffff89b545108380 x1855007616828928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.598064:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108380 with x1855007616828928 ext(2498560->2502655) 00002000:00010000:1.0:1769074771.598067:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545108380 00000100:00100000:3.0:1769074771.612051:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.617450:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829056 00000100:00100000:2.0:1769074771.617471:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829056 00000100:00100000:2.0:1769074771.618064:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14038 00000100:00100000:2.0:1769074771.618079:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62829:x1855007616829056:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074771.618104:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074771.618130:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb66 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.618170:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.618180:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 36 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.618187:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.618189:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.618193:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3448831] (req 3145728->3448831) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.618231:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 36 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.618244:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 36 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.618257:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625133696:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074771.618393:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.618403:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 22 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.618411:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625133824:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074771.618434:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.620961:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625133696:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074771.631929:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.632021:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.632178:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829184 00000100:00100000:3.0:1769074771.632189:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829184 00010000:00010000:3.0:1769074771.632232:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 37 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.632280:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829312 00000100:00100000:3.0:1769074771.632283:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829312 00010000:00010000:3.0:1769074771.632296:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 23 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.632316:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7007 00000100:00100000:3.0:1769074771.632338:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2409:x1855007616829184:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.632354:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074771.632375:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 37 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.632388:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 37 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.632410:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd5394a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb43 expref: 37 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.632424:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074771.632476:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616829184:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 151us (559us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074771.632484:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7007 00000100:00100000:3.0:1769074771.632500:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7008 00000100:00100000:3.0:1769074771.632505:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2409:x1855007616829312:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.632510:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074771.632516:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 23 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.632533:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 23 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.632543:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.632544:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.632550:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.632557:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.632583:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.632594:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625133952:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074771.632634:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00010000:00010000:2.0:1769074771.633829:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd5393c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb35 expref: 23 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.633849:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625133824:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074771.633879:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.633894:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b4000, rc 0) 00000100:00100000:2.0:1769074771.633944:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62829:x1855007616829056:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 15870us (21905us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074771.633954:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14038 00000100:00100000:3.0:1769074771.641788:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625133952:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074771.641884:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.642057:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616829312:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9548us (10033us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.642068:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7008 00000100:00100000:3.0:1769074771.652547:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.654040:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829440 00000100:00100000:1.0:1769074771.654054:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829440 00002000:00100000:1.0:1769074771.654109:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3444736->3448831) req@ffff89b545109880 x1855007616829440/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.654129:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109880 with x1855007616829440 ext(3444736->3448831) 00002000:00010000:1.0:1769074771.654518:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545109880 00000100:00100000:1.0:1769074771.654640:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8529 00000100:00100000:1.0:1769074771.654652:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616829440:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074771.654906:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616829440:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 256us (2370us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074771.654913:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8529 00002000:00100000:1.0:1769074771.654920:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3444736->3448831) req@ffff89b545109880 x1855007616829440/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.654929:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109880 with x1855007616829440 ext(3444736->3448831) 00002000:00010000:1.0:1769074771.654933:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545109880 00000100:00100000:3.0:1769074771.688015:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.688469:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829568 00000100:00100000:1.0:1769074771.688479:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829568 00002000:00100000:1.0:1769074771.688522:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54510b100 x1855007616829568/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074771.688530:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510b100 with x1855007616829568 ext(1048576->2097151) 00002000:00010000:1.0:1769074771.688535:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54510b100 00000100:00100000:1.0:1769074771.688564:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8530 00000100:00100000:1.0:1769074771.688571:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616829568:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074771.693047:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.697037:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829696 00000100:00100000:2.0:1769074771.697058:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829696 00002000:00100000:2.0:1769074771.697123:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b549f29f80 x1855007616829696/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.697359:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f29f80 with x1855007616829696 ext(1048576->2097151) 00002000:00010000:2.0:1769074771.697366:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f29f80 00000100:00100000:2.0:1769074771.697403:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8531 00000100:00100000:2.0:1769074771.697413:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616829696:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074771.708898:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616829568:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20309us (20877us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074771.708918:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8530 00002000:00100000:0.0:1769074771.708934:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54510b100 x1855007616829568/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.708944:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510b100 with x1855007616829568 ext(1048576->2097151) 00002000:00010000:0.0:1769074771.708950:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54510b100 00000100:00100000:2.0:1769074771.713652:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616829696:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16228us (20606us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074771.713680:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8531 00002000:00100000:2.0:1769074771.713694:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b549f29f80 x1855007616829696/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.713705:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f29f80 with x1855007616829696 ext(1048576->2097151) 00002000:00010000:2.0:1769074771.713711:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f29f80 00000100:00100000:0.0:1769074771.740490:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.740628:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.740976:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829824 00000100:00100000:2.0:1769074771.740987:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829824 00002000:00100000:2.0:1769074771.741068:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54ce9ea00 x1855007616829824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 504/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.741087:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ea00 with x1855007616829824 ext(2097152->4194303) 00002000:00010000:2.0:1769074771.741095:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9ea00 00000100:00100000:1.0:1769074771.741552:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616829952 00000100:00100000:1.0:1769074771.741559:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616829952 00002000:00100000:0.0:1769074771.741662:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54ce9e300 x1855007616829952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 504/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.741683:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9e300 with x1855007616829952 ext(2097152->4194303) 00002000:00010000:0.0:1769074771.741688:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9e300 00000100:00100000:0.0:1769074771.741720:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8532 00000100:00100000:0.0:1769074771.741730:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616829824:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.742163:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8533 00000100:00100000:1.0:1769074771.742173:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616829952:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074771.764940:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616829952:12345-192.168.204.60@tcp:3:cat.0 Request processed in 22756us (24300us total) trans 0 rc 1449984/1449984 00000100:00100000:2.0:1769074771.764955:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8533 00002000:00100000:2.0:1769074771.764967:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54ce9e300 x1855007616829952/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 504/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1449984/1449984 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.764977:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9e300 with x1855007616829952 ext(2097152->4194303) 00002000:00010000:2.0:1769074771.764984:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9e300 00000100:00100000:0.0:1769074771.793887:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616829824:12345-192.168.204.60@tcp:3:cat.0 Request processed in 52150us (53400us total) trans 0 rc 1794048/1794048 00000100:00100000:0.0:1769074771.793898:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8532 00002000:00100000:0.0:1769074771.793909:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54ce9ea00 x1855007616829824/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 504/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 1794048/1794048 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.793916:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ea00 with x1855007616829824 ext(2097152->4194303) 00002000:00010000:0.0:1769074771.793921:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9ea00 00000100:00100000:1.0:1769074771.802988:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.803101:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616830080 00000100:00100000:2.0:1769074771.803113:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616830080 00000100:00100000:2.0:1769074771.803204:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14039 00000100:00100000:2.0:1769074771.803214:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62831:x1855007616830080:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074771.803240:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074771.803258:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb6d expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.803293:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.803304:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 11 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074771.803312:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.803315:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.803319:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1859584->2097151] (req 1859584->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.803359:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 11 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074771.803373:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 11 pid: 28954 timeout: 1859 lvb_type: 1 00000100:00100000:2.0:1769074771.803387:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134080:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074771.803475:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.803484:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.803493:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134208:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074771.803514:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074771.817721:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.817999:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616830592 00000100:00100000:2.0:1769074771.818009:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616830592 00000100:00100000:2.0:1769074771.818072:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13316 00000100:00100000:2.0:1769074771.818081:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62841:x1855007616830592:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074771.818104:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074771.818123:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd53997 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb74 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.818198:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818213:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818225:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818249:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818256:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818262:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818557:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd539ac lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818569:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd539ac lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818575:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd539ac lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074771.818584:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074771.818653:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818660:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818743:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818750:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818757:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b6725b5800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074771.818761:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818768:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818781:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074771.818785:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074771.818789:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818796:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5800/0x7fe2323adfd5399e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818863:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818870:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818875:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818879:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b546117000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074771.818882:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818892:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818899:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4400) 00010000:00010000:2.0:1769074771.818901:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074771.818903:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818909:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117000/0x7fe2323adfd539a5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074771.818948:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5600/0x7fe2323adfd53997 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb74 expref: 246 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.818961:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd539ac lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb74 expref: 245 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.818968:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074771.818970:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074771.818973:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602b800, rc 0) 00000100:00100000:2.0:1769074771.819036:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62841:x1855007616830592:12345-192.168.204.60@tcp:101:cat.0 Request processed in 957us (1328us total) trans 8589943939 rc 0/0 00000100:00100000:2.0:1769074771.819044:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13316 00000100:00100000:3.0:1769074771.819797:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.820493:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616830208 00000100:00100000:0.0:1769074771.820502:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616830208 00010000:00010000:0.0:1769074771.820536:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 12 pid: 28954 timeout: 1859 lvb_type: 1 00000100:00100000:0.0:1769074771.820571:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7009 00000100:00100000:0.0:1769074771.820579:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616830208:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.820591:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.820612:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 12 pid: 28954 timeout: 1859 lvb_type: 1 00010000:00010000:0.0:1769074771.820627:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 12 pid: 28954 timeout: 1859 lvb_type: 1 00010000:00010000:0.0:1769074771.820649:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.820771:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616830208:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 193us (979us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.820778:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7009 00010000:00010000:0.0:1769074771.820850:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53951 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb4a expref: 11 pid: 28954 timeout: 1859 lvb_type: 1 00000100:00100000:0.0:1769074771.820865:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134080:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074771.820876:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134208:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074771.820905:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1859584->2097151] (req 1859584->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.820917:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5200, rc 0) 00000100:00100000:0.0:1769074771.820938:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62831:x1855007616830080:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 17727us (17961us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.820946:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14039 00000100:00100000:3.0:1769074771.822265:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.822568:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616830720 00000100:00100000:0.0:1769074771.822578:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616830720 00002000:00100000:0.0:1769074771.822622:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67ff0dc00 x1855007616830720/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074771.822633:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0dc00 with x1855007616830720 ext(4194304->4808703) 00002000:00010000:0.0:1769074771.822639:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0dc00 00000100:00100000:0.0:1769074771.822707:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8534 00000100:00100000:0.0:1769074771.822716:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616830720:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074771.825913:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.826435:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.827976:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616830336 00000100:00100000:3.0:1769074771.827990:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616830336 00010000:00010000:3.0:1769074771.828040:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.828165:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7010 00000100:00100000:3.0:1769074771.828176:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616830336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.828192:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074771.828219:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.828246:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.828285:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866600/0x7fe2323adfd53912 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb0b expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:3.0:1769074771.828304:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.828307:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.828316:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.828343:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.828385:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.828407:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625134336:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074771.828505:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e9843c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074771.828548:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616830976 00000100:00100000:3.0:1769074771.828553:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616830976 00000100:00100000:3.0:1769074771.828584:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4664 00000100:00100000:3.0:1769074771.828591:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62841:x1855007616830976:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074771.828777:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+246:62841:x1855007616830976:12345-192.168.204.60@tcp:35:cat.0 Request processed in 186us (2343us total) trans 8589943940 rc 0/0 00000100:00100000:3.0:1769074771.828785:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4664 00000100:00100000:1.0:1769074771.831083:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074771.831158:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831104 00000100:00100000:0.0:1769074771.831168:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831104 00000100:00100000:0.0:1769074771.831250:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1006 00000100:00100000:0.0:1769074771.831261:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b682063850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+246:2408:x1855007616831104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074771.831289:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074771.831318:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd539ac lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb74 expref: 246 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074771.831346:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074771.831351:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074771.831404:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b682063850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:2408:x1855007616831104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 145us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074771.831411:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1006 00000100:00100000:2.0:1769074771.833967:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616830720:12345-192.168.204.60@tcp:3:cat.0 Request processed in 11241us (11698us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074771.833981:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8534 00002000:00100000:2.0:1769074771.833994:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67ff0dc00 x1855007616830720/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.834011:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0dc00 with x1855007616830720 ext(4194304->4808703) 00002000:00010000:2.0:1769074771.834017:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0dc00 00000100:00100000:3.0:1769074771.834991:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625134336:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074771.835015:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074771.835938:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616830336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7757us (10028us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074771.835950:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7010 00000100:00100000:1.0:1769074771.840756:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.840799:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.840849:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074771.840872:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831360 00000100:00100000:1.0:1769074771.840882:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831360 00000100:00100000:1.0:1769074771.840923:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831488 00000100:00100000:1.0:1769074771.840925:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831488 00000100:00100000:1.0:1769074771.840949:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14040 00000100:00100000:1.0:1769074771.840958:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616831360:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074771.840974:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074771.840991:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd539ba lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb7b expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074771.841064:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625134464:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074771.841125:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074771.841144:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14041 00000100:00100000:1.0:1769074771.841150:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616831488:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074771.841157:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074771.841166:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd539c1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb82 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074771.841190:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625134592:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074771.841210:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b26f600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.841307:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831232 00000100:00100000:2.0:1769074771.841312:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831232 00002000:00100000:2.0:1769074771.841350:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1859584->1863679) req@ffff89b54cde4700 x1855007616831232/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.841360:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4700 with x1855007616831232 ext(1859584->1863679) 00002000:00010000:2.0:1769074771.841365:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde4700 00000100:00100000:2.0:1769074771.841386:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8535 00000100:00100000:2.0:1769074771.841392:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616831232:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074771.841528:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616831232:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 136us (783us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074771.841536:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8535 00002000:00100000:2.0:1769074771.841542:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1859584->1863679) req@ffff89b54cde4700 x1855007616831232/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.841551:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cde4700 with x1855007616831232 ext(1859584->1863679) 00002000:00010000:2.0:1769074771.841555:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cde4700 00000100:00100000:1.0:1769074771.842977:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.843042:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831616 00000100:00100000:3.0:1769074771.843050:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831616 00000100:00100000:3.0:1769074771.843093:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14042 00000100:00100000:3.0:1769074771.843100:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616831616:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074771.843115:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074771.843130:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd539c8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb89 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074771.843191:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134720:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074771.843543:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00002000:00010000:2.0:1769074771.852286:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074770 -> 1769074771 00002000:00010000:2.0:1769074771.852299:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074769 -> 1769074771 00002000:00010000:2.0:1769074771.852302:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074770 -> 1769074771 00000100:00100000:2.0:1769074771.852337:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625134464:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074771.852383:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd539ba lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb7b expref: 37 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074771.852412:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd539ba lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb7b expref: 37 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074771.852426:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:2.0:1769074771.852495:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616831360:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11536us (11693us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074771.852506:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14040 00002000:00010000:3.0:1769074771.854533:0:8398:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074770 -> 1769074771 00002000:00010000:3.0:1769074771.854542:0:8398:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074769 -> 1769074771 00002000:00010000:3.0:1769074771.854544:0:8398:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074770 -> 1769074771 00000100:00100000:3.0:1769074771.854566:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625134592:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074771.854602:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd539c1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb82 expref: 18 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.854623:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd539c1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb82 expref: 18 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.854636:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:2.0:1769074771.858621:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616831488:12345-192.168.204.60@tcp:101:cat.0 Request processed in 17456us (17756us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074771.858636:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14041 00000100:00100000:1.0:1769074771.867371:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:3.0:1769074771.867882:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074770 -> 1769074771 00002000:00010000:3.0:1769074771.867888:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074770 -> 1769074771 00002000:00010000:3.0:1769074771.867891:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074770 -> 1769074771 00000100:00100000:3.0:1769074771.867915:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134720:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074771.867955:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd539c8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb89 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.867979:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd539c8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb89 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.867992:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074771.868065:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616831616:12345-192.168.204.60@tcp:101:cat.0 Request processed in 24955us (25087us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074771.868074:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14042 00000100:00100000:3.0:1769074771.868114:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831744 00000100:00100000:3.0:1769074771.868123:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831744 00000100:00100000:3.0:1769074771.868184:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14043 00000100:00100000:3.0:1769074771.868193:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62831:x1855007616831744:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074771.868212:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074771.868233:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb90 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.868273:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.868283:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 10 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.868292:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.868295:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074771.868299:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1810431] (req 1048576->1810431) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.868344:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 10 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.868358:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 10 pid: 28954 timeout: 1859 lvb_type: 1 00000100:00100000:3.0:1769074771.868372:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134848:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074771.868436:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.868445:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074771.868455:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134976:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074771.868476:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:3.0:1769074771.874659:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134848:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074771.874763:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625134976:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074771.874800:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->1810431] (req 1048576->1810431) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074771.874867:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:3.0:1769074771.874971:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62831:x1855007616831744:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6777us (7613us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074771.874980:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14043 00000100:00100000:0.0:1769074771.877587:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.877763:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616831872 00000100:00100000:3.0:1769074771.877774:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616831872 00010000:00010000:3.0:1769074771.878468:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 11 pid: 28954 timeout: 1859 lvb_type: 1 00000100:00100000:3.0:1769074771.878527:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7011 00000100:00100000:3.0:1769074771.878540:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616831872:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074771.878559:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074771.878582:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 11 pid: 28954 timeout: 1859 lvb_type: 1 00010000:00010000:3.0:1769074771.878597:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 11 pid: 28954 timeout: 1859 lvb_type: 1 00010000:00010000:3.0:1769074771.878622:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53958 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb51 expref: 11 pid: 28954 timeout: 1859 lvb_type: 1 00010000:00010000:3.0:1769074771.878641:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074771.884213:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616831872:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5664us (6624us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074771.884234:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7011 00000100:00100000:3.0:1769074771.884432:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.884545:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832000 00000100:00100000:2.0:1769074771.884554:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832000 00010000:00010000:2.0:1769074771.884586:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074771.884623:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7012 00000100:00100000:2.0:1769074771.884633:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616832000:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074771.884647:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074771.884673:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074771.884692:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074771.884716:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd53919 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb12 expref: 18 pid: 28954 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074771.884731:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.884733:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074771.884740:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.884749:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074771.884787:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074771.884803:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625135104:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074771.884915:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074771.898920:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.899023:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074771.899629:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.899774:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832384 00000100:00100000:2.0:1769074771.899787:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832384 00002000:00100000:2.0:1769074771.899874:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1859583) req@ffff89b67ff0d180 x1855007616832384/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.899886:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0d180 with x1855007616832384 ext(1048576->1859583) 00002000:00010000:2.0:1769074771.899893:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0d180 00000100:00100000:2.0:1769074771.899925:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8536 00000100:00100000:2.0:1769074771.899934:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2407:x1855007616832384:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074771.908145:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625135104:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074771.908171:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074771.908232:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616832000:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 23599us (23800us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074771.908243:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7012 00000100:00100000:0.0:1769074771.909384:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832256 00000100:00100000:0.0:1769074771.909395:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832256 00000100:00100000:0.0:1769074771.909448:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832128 00000100:00100000:0.0:1769074771.909450:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832128 00000100:00100000:0.0:1769074771.909524:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14044 00000100:00100000:0.0:1769074771.909533:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616832256:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074771.909551:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074771.909568:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641521e00/0x7fe2323adfd539d6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb9e expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074771.909633:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625135232:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074771.909714:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074771.909929:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14045 00000100:00100000:1.0:1769074771.909938:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616832128:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074771.909945:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074771.909954:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd539dd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bb97 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074771.909981:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625135360:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074771.910016:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b26f600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074771.925885:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616832384:12345-192.168.204.60@tcp:3:cat.0 Request processed in 25936us (26241us total) trans 0 rc 811008/811008 00000100:00100000:2.0:1769074771.925900:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8536 00002000:00100000:2.0:1769074771.925915:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->1859583) req@ffff89b67ff0d180 x1855007616832384/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 811008/811008 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.925925:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0d180 with x1855007616832384 ext(1048576->1859583) 00002000:00010000:2.0:1769074771.925932:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0d180 00000100:00100000:0.0:1769074771.947221:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:3.0:1769074771.947349:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074770 -> 1769074771 00002000:00010000:3.0:1769074771.947354:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074769 -> 1769074771 00002000:00010000:3.0:1769074771.947357:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074770 -> 1769074771 00000100:00100000:3.0:1769074771.947380:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625135232:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074771.947428:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641521e00/0x7fe2323adfd539d6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb9e expref: 13 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.947457:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641521e00/0x7fe2323adfd539d6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb9e expref: 13 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074771.947476:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521e00, rc 0) 00000100:00100000:3.0:1769074771.947547:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616832256:12345-192.168.204.60@tcp:101:cat.0 Request processed in 38014us (48636us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074771.947557:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14044 00000100:00100000:1.0:1769074771.948260:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625135360:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074771.948296:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd539dd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb97 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074771.948310:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd539dd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb97 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074771.948328:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:1.0:1769074771.948368:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616832128:12345-192.168.204.60@tcp:101:cat.0 Request processed in 38431us (49344us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074771.948375:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14045 00000100:00100000:2.0:1769074771.948872:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832512 00000100:00100000:2.0:1769074771.948881:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832512 00002000:00100000:2.0:1769074771.948918:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1806336->1810431) req@ffff89b54ce9ed80 x1855007616832512/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.948926:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ed80 with x1855007616832512 ext(1806336->1810431) 00002000:00010000:2.0:1769074771.948930:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9ed80 00000100:00100000:2.0:1769074771.948975:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8537 00000100:00100000:2.0:1769074771.948981:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616832512:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074771.949155:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616832512:12345-192.168.204.60@tcp:3:cat.0 Request processed in 173us (1944us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074771.949162:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8537 00002000:00100000:2.0:1769074771.949168:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1806336->1810431) req@ffff89b54ce9ed80 x1855007616832512/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/4536 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.949176:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ed80 with x1855007616832512 ext(1806336->1810431) 00002000:00010000:2.0:1769074771.949180:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9ed80 00000100:00100000:3.0:1769074771.972449:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074771.972849:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832640 00000100:00100000:2.0:1769074771.972862:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832640 00002000:00100000:2.0:1769074771.972920:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1810432->2097151) req@ffff89b67ff0d880 x1855007616832640/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/0 e 0 to 0 dl 1769074782 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.972931:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0d880 with x1855007616832640 ext(1810432->2097151) 00002000:00010000:2.0:1769074771.972937:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0d880 00000100:00100000:2.0:1769074771.972977:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8538 00000100:00100000:2.0:1769074771.972986:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616832640:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074771.977953:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616832640:12345-192.168.204.60@tcp:3:cat.0 Request processed in 4955us (5503us total) trans 0 rc 286720/286720 00000100:00100000:2.0:1769074771.977967:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8538 00002000:00100000:2.0:1769074771.977979:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bb:0x0] (1810432->2097151) req@ffff89b67ff0d880 x1855007616832640/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:307/0 lens 488/440 e 0 to 0 dl 1769074782 ref 1 fl Complete:/600/0 rc 286720/286720 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074771.977991:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0d880 with x1855007616832640 ext(1810432->2097151) 00002000:00010000:2.0:1769074771.977998:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0d880 00000100:00100000:0.0:1769074772.007597:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.007651:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.008591:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832768 00000100:00100000:1.0:1769074772.008602:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832768 00000100:00100000:1.0:1769074772.008670:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616832896 00000100:00100000:1.0:1769074772.008674:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616832896 00000100:00100000:1.0:1769074772.008729:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14046 00000100:00100000:1.0:1769074772.008738:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2407:x1855007616832768:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.008757:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.008779:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fe00/0x7fe2323adfd539e4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bba5 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.008871:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625135488:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074772.008949:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b26f600 going to sleep for 11 seconds 00000100:00100000:1.0:1769074772.008974:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14047 00000100:00100000:1.0:1769074772.008979:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616832896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.008984:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.008992:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd539eb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbac expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.009011:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625135616:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074772.009027:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ea80 going to sleep for 11 seconds 00000100:00080000:0.0:1769074772.009171:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074772.014582:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625135488:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074772.014631:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fe00/0x7fe2323adfd539e4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bba5 expref: 37 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074772.014653:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fe00/0x7fe2323adfd539e4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bba5 expref: 37 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074772.014675:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:1.0:1769074772.014741:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2407:x1855007616832768:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6004us (7157us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074772.014751:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14046 00000100:00100000:2.0:1769074772.022139:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625135616:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074772.022183:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd539eb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbac expref: 18 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.022212:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd539eb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbac expref: 18 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.022229:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e400, rc 0) 00000100:00100000:2.0:1769074772.022909:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2406:x1855007616832896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13923us (15251us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074772.022921:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14047 00000100:00100000:3.0:1769074772.045913:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.046017:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833024 00000100:00100000:2.0:1769074772.046027:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833024 00000100:00100000:2.0:1769074772.046086:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14048 00000100:00100000:2.0:1769074772.046096:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:62841:x1855007616833024:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.046113:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.046131:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbb3 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.046170:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.046179:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.046181:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.046185:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.046217:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.046231:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 20 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:2.0:1769074772.046245:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625135744:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074772.046332:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6501c7540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.050200:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625135744:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074772.050241:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.050263:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:2.0:1769074772.050333:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:62841:x1855007616833024:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4238us (4428us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.050342:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14048 00000100:00100000:0.0:1769074772.052890:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.053147:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833152 00000100:00100000:2.0:1769074772.053157:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833152 00002000:00100000:2.0:1769074772.053203:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3448831) req@ffff89b54ce9ca80 x1855007616833152/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.053213:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ca80 with x1855007616833152 ext(3145728->3448831) 00010000:00010000:2.0:1769074772.053222:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 21 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074772.053236:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 21 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:2.0:1769074772.053244:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9ca80 00000100:00100000:2.0:1769074772.053267:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 759 00000100:00100000:2.0:1769074772.053277:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616833152:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074772.067944:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616833152:12345-192.168.204.60@tcp:4:cat.0 Request processed in 14658us (15052us total) trans 17179874508 rc 0/0 00000100:00100000:2.0:1769074772.067958:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 759 00002000:00100000:2.0:1769074772.067970:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3448831) req@ffff89b54ce9ca80 x1855007616833152/t17179874508(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.067981:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ca80 with x1855007616833152 ext(3145728->3448831) 00010000:00010000:2.0:1769074772.067991:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 23 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:2.0:1769074772.068009:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 23 pid: 8396 timeout: 1859 lvb_type: 0 00002000:00010000:2.0:1769074772.068018:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9ca80 00000001:00080000:0.0:1769074772.068424:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874507 is committed 00000001:00080000:0.0:1769074772.068446:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874504 is committed 00000100:00100000:3.0:1769074772.086607:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.086643:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.086832:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833408 00000100:00100000:2.0:1769074772.086841:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833408 00000100:00100000:2.0:1769074772.086894:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14049 00000100:00100000:2.0:1769074772.086903:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62842:x1855007616833408:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074772.086921:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.086939:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbba expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.086966:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.086968:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.086973:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3891199] (req 3145728->3891199) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.086984:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3891199] (req 3145728->3891199) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.086995:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867200, rc 0) 00000100:00100000:0.0:1769074772.087147:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833280 00000100:00100000:0.0:1769074772.087152:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833280 00010000:00010000:0.0:1769074772.087177:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:0.0:1769074772.087204:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7013 00000100:00100000:0.0:1769074772.087211:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616833280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.087221:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074772.087244:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074772.087256:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074772.087278:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b4000/0x7fe2323adfd53989 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3448831) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb66 expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:0.0:1769074772.087297:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.087299:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.087306:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.087316:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.087332:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.087334:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.087337:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.087343:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.087373:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074772.087387:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625135872:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074772.088279:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62842:x1855007616833408:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1371us (1676us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.088407:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14049 00010000:00010000:0.0:1769074772.088757:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074772.088773:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625136000:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074772.088832:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.098796:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.099244:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833536 00000100:00100000:1.0:1769074772.099256:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833536 00002000:00100000:1.0:1769074772.099334:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b67ff0f800 x1855007616833536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074772.099346:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0f800 with x1855007616833536 ext(2097152->3145727) 00002000:00010000:1.0:1769074772.099352:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0f800 00000100:00100000:1.0:1769074772.099385:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8539 00000100:00100000:1.0:1769074772.099394:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616833536:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074772.100071:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625135872:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074772.100086:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625136000:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074772.100111:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.102838:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616833280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 15614us (16179us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.102853:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7013 00000100:00100000:3.0:1769074772.120906:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.121937:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833664 00000100:00100000:2.0:1769074772.121957:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833664 00002000:00100000:2.0:1769074772.122028:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3887104->3891199) req@ffff89b67ff0ed80 x1855007616833664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.122040:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ed80 with x1855007616833664 ext(3887104->3891199) 00002000:00010000:2.0:1769074772.122047:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0ed80 00000100:00100000:2.0:1769074772.122099:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8540 00000100:00100000:2.0:1769074772.122111:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616833664:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074772.122876:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616833664:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 762us (1976us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074772.122888:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8540 00002000:00100000:2.0:1769074772.122897:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3887104->3891199) req@ffff89b67ff0ed80 x1855007616833664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.122907:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ed80 with x1855007616833664 ext(3887104->3891199) 00002000:00010000:2.0:1769074772.122912:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0ed80 00000100:00100000:1.0:1769074772.131958:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616833536:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32554us (33161us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074772.131971:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8539 00002000:00100000:1.0:1769074772.131982:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b67ff0f800 x1855007616833536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074772.131993:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0f800 with x1855007616833536 ext(2097152->3145727) 00002000:00010000:1.0:1769074772.131998:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0f800 00000100:00100000:1.0:1769074772.152220:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.152350:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.152550:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.154886:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833792 00000100:00100000:2.0:1769074772.154908:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833792 00000100:00100000:2.0:1769074772.155102:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616833920 00000100:00100000:2.0:1769074772.155106:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616833920 00000100:00100000:3.0:1769074772.155678:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834048 00000100:00100000:3.0:1769074772.155684:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834048 00000100:00100000:3.0:1769074772.155723:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14050 00000100:00100000:3.0:1769074772.155732:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616833792:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.155751:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.155769:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53a00 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbc1 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.155845:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625136128:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074772.155923:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.155944:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14051 00000100:00100000:3.0:1769074772.155951:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616833920:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.155958:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.155966:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a07 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbc8 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.155991:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625136256:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074772.156014:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.156030:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14052 00000100:00100000:3.0:1769074772.156038:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616834048:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.156046:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.156055:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd53a0e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbcf expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.156079:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625136384:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074772.156104:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.156263:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.156293:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834176 00000100:00100000:3.0:1769074772.156297:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834176 00000100:00100000:3.0:1769074772.156342:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14053 00000100:00100000:3.0:1769074772.156350:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62841:x1855007616834176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.156357:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.156366:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbd6 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.156384:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.156392:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.156394:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.156398:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.156421:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.156435:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 1859 lvb_type: 0 00000100:00100000:3.0:1769074772.156460:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625136512:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074772.156484:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c510c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.160726:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625136384:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074772.160764:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd53a0e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbcf expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.160794:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910da00/0x7fe2323adfd53a0e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbcf expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.160843:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:2.0:1769074772.160903:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616834048:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4865us (8353us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074772.160912:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14052 00000100:00100000:0.0:1769074772.161646:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625136256:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074772.161679:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a07 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbc8 expref: 13 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074772.161695:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a07 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbc8 expref: 13 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074772.161705:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:0.0:1769074772.161748:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616833920:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5798us (9398us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074772.161756:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14051 00000100:00100000:1.0:1769074772.161958:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625136128:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074772.161974:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53a00 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbc1 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074772.161993:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53a00 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbc1 expref: 11 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074772.162001:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:1.0:1769074772.162024:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616833792:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6293us (9891us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074772.162030:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14050 00000100:00100000:3.0:1769074772.163962:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.164065:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625136512:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074772.164098:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 18 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.164116:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d200, rc 0) 00000100:00100000:0.0:1769074772.164169:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:62841:x1855007616834176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7818us (7905us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.164178:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14053 00000100:00100000:1.0:1769074772.164368:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834304 00000100:00100000:1.0:1769074772.164376:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834304 00002000:00100000:1.0:1769074772.164419:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2498560->3145727) req@ffff89b543858700 x1855007616834304/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074772.164428:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858700 with x1855007616834304 ext(2498560->3145727) 00010000:00010000:1.0:1769074772.164435:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 10 pid: 8396 timeout: 1859 lvb_type: 0 00010000:00010000:1.0:1769074772.164447:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 10 pid: 8396 timeout: 1860 lvb_type: 0 00002000:00010000:1.0:1769074772.164454:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b543858700 00000100:00100000:1.0:1769074772.164478:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 760 00000100:00100000:1.0:1769074772.164487:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616834304:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074772.171997:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.177442:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616834304:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 12942us (13475us total) trans 21474841194 rc 0/0 00000100:00100000:2.0:1769074772.177458:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 760 00002000:00100000:2.0:1769074772.177472:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2498560->3145727) req@ffff89b543858700 x1855007616834304/t21474841194(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.177483:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858700 with x1855007616834304 ext(2498560->3145727) 00010000:00010000:2.0:1769074772.177494:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 12 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.177511:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 12 pid: 8396 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.177520:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b543858700 00000100:00100000:1.0:1769074772.177535:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834432 00000100:00100000:1.0:1769074772.177545:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834432 00000100:00100000:1.0:1769074772.177596:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14054 00000100:00100000:1.0:1769074772.177606:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62847:x1855007616834432:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.177623:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.177641:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbdd expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.177883:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.177893:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074772.177895:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074772.177902:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.177937:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.177949:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 17 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.178018:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625136640:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074772.178148:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.183838:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.183891:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625136640:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074772.183926:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.183947:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e000, rc 0) 00000100:00100000:1.0:1769074772.184007:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62847:x1855007616834432:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6402us (12021us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.184017:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14054 00000100:00100000:2.0:1769074772.184494:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834560 00000100:00100000:2.0:1769074772.184503:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834560 00002000:00100000:2.0:1769074772.184550:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1859584->2097151) req@ffff89b54ce9df80 x1855007616834560/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.184578:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9df80 with x1855007616834560 ext(1859584->2097151) 00010000:00010000:2.0:1769074772.184587:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 18 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.184599:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 18 pid: 8396 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.184606:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54ce9df80 00000100:00100000:2.0:1769074772.184641:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 761 00000100:00100000:2.0:1769074772.184650:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616834560:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074772.192095:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.192351:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834688 00000100:00100000:3.0:1769074772.192362:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834688 00010000:00010000:3.0:1769074772.192407:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.192454:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7014 00000100:00100000:3.0:1769074772.192465:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616834688:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074772.192480:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074772.192503:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.192517:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.192543:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b4a00/0x7fe2323adfd53982 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2498560->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb5f expref: 13 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.192878:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.192882:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.192892:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.192904:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.192945:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.192962:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625136768:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074772.193426:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c510c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.203732:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625136768:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074772.203759:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074772.203858:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616834688:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11394us (11774us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.203869:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7014 00000100:00100000:2.0:1769074772.204740:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616834560:12345-192.168.204.60@tcp:4:cat.0 Request processed in 19909us (20764us total) trans 21474841195 rc 0/0 00000100:00100000:2.0:1769074772.204752:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 761 00002000:00100000:2.0:1769074772.204764:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1859584->2097151) req@ffff89b54ce9df80 x1855007616834560/t21474841195(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.204775:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9df80 with x1855007616834560 ext(1859584->2097151) 00010000:00010000:2.0:1769074772.204784:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 20 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.204798:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 20 pid: 8396 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.204837:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54ce9df80 00000100:00100000:0.0:1769074772.216127:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.216292:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834816 00000100:00100000:2.0:1769074772.216305:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834816 00002000:00100000:2.0:1769074772.216370:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54ce9d500 x1855007616834816/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.216380:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9d500 with x1855007616834816 ext(2097152->3145727) 00002000:00010000:2.0:1769074772.216385:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9d500 00000100:00100000:2.0:1769074772.216419:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8541 00000100:00100000:2.0:1769074772.216428:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616834816:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074772.227889:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.231454:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616834944 00000100:00100000:3.0:1769074772.231470:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616834944 00010000:00010000:3.0:1769074772.231905:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 22 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.232368:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7015 00000100:00100000:1.0:1769074772.232384:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616834944:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074772.232400:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074772.232425:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 22 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.232450:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 22 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.232474:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b6725b5200/0x7fe2323adfd53990 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1859584->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb6d expref: 22 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.232492:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.232494:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.232502:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.232513:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.232553:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.232576:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625136896:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074772.232678:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.243483:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625136896:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074772.243513:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074772.243595:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616834944:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11217us (15719us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.243607:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7015 00000100:00100000:1.0:1769074772.243893:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.244011:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616835072 00000100:00100000:2.0:1769074772.244018:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616835072 00002000:00100000:2.0:1769074772.244061:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5497a8700 x1855007616835072/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.244072:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8700 with x1855007616835072 ext(1048576->2097151) 00002000:00010000:2.0:1769074772.244077:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497a8700 00000100:00100000:2.0:1769074772.244108:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8542 00000100:00100000:2.0:1769074772.244117:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616835072:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074772.261570:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616834816:12345-192.168.204.60@tcp:3:cat.0 Request processed in 45131us (45445us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074772.261584:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8541 00002000:00100000:2.0:1769074772.261597:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b54ce9d500 x1855007616834816/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.261608:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9d500 with x1855007616834816 ext(2097152->3145727) 00002000:00010000:2.0:1769074772.261615:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9d500 00000100:00100000:2.0:1769074772.276975:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616835072:12345-192.168.204.60@tcp:3:cat.0 Request processed in 32847us (33071us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074772.276993:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8542 00002000:00100000:2.0:1769074772.277007:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5497a8700 x1855007616835072/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.277019:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497a8700 with x1855007616835072 ext(1048576->2097151) 00002000:00010000:2.0:1769074772.277026:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5497a8700 00000100:00100000:2.0:1769074772.293974:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.294020:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.294940:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616835328 00000100:00100000:0.0:1769074772.294954:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616835328 00000100:00100000:0.0:1769074772.295037:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13317 00000100:00100000:0.0:1769074772.295048:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62850:x1855007616835328:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074772.295072:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074772.295092:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53a23 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbe4 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.295184:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295199:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295211:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295226:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295232:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295238:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295365:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53a38 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295377:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53a38 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295384:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53a38 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:0.0:1769074772.295393:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074772.295481:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295488:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295494:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295499:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295505:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074772.295509:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295516:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295526:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074772.295530:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074772.295534:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295540:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9400/0x7fe2323adfd53a2a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.295557:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296011:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296018:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296023:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074772.296027:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296032:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296041:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47400) 00010000:00010000:0.0:1769074772.296043:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074772.296046:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296052:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8800/0x7fe2323adfd53a31 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074772.296099:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53a23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbe4 expref: 248 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.296112:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53a38 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbe4 expref: 247 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.296119:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074772.296123:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074772.296125:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:0.0:1769074772.296204:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62850:x1855007616835328:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1158us (2240us total) trans 8589943941 rc 0/0 00000100:00100000:0.0:1769074772.296211:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13317 00000100:00080000:2.0:1769074772.296605:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074772.296836:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616835456 00000100:00100000:1.0:1769074772.296844:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616835456 00000100:00100000:1.0:1769074772.296893:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14055 00000100:00100000:1.0:1769074772.296902:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62847:x1855007616835456:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.296915:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.296929:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbeb expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.296953:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.296961:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074772.296963:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074772.296967:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.296999:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.297012:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 18 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.297025:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5497aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625137024:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074772.297084:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.308832:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.308880:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.308936:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.308977:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.309015:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836096 00000100:00100000:2.0:1769074772.309025:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836096 00002000:00100000:2.0:1769074772.309074:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1810431) req@ffff89b549203480 x1855007616836096/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.309086:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203480 with x1855007616836096 ext(1048576->1810431) 00010000:00010000:2.0:1769074772.309096:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 19 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.309345:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616835968 00000100:00100000:3.0:1769074772.309350:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616835968 00000100:00100000:3.0:1769074772.309391:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4665 00000100:00100000:3.0:1769074772.309400:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62850:x1855007616835968:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074772.309996:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62850:x1855007616835968:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 595us (1058us total) trans 8589943942 rc 0/0 00000100:00100000:3.0:1769074772.310004:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4665 00000100:00100000:3.0:1769074772.310031:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616835584 00000100:00100000:3.0:1769074772.310035:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616835584 00000100:00100000:3.0:1769074772.310057:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14056 00000100:00100000:3.0:1769074772.310063:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62841:x1855007616835584:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.310073:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.310091:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbf2 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.310218:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.310228:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.310230:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.310235:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.310266:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.310278:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 10 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.310292:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625137152:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074772.310472:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616835712 00000100:00100000:1.0:1769074772.310478:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616835712 00000100:00100000:1.0:1769074772.310516:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1007 00000100:00100000:1.0:1769074772.310523:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:2408:x1855007616835712:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074772.310533:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074772.310554:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53a38 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbe4 expref: 248 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.310581:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074772.310587:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074772.310613:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:2408:x1855007616835712:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 90us (1732us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.310620:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1007 00000100:00100000:1.0:1769074772.310668:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5497aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625137024:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074772.310691:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 13 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.310704:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:1.0:1769074772.310725:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62847:x1855007616835456:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13824us (16705us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.310731:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14055 00000100:00100000:3.0:1769074772.310964:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00010000:00010000:2.0:1769074772.311915:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 19 pid: 8396 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.311927:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549203480 00000100:00100000:2.0:1769074772.311956:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 762 00000100:00100000:2.0:1769074772.311968:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616836096:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074772.320272:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.320711:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625137152:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074772.320742:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.320760:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:2.0:1769074772.320849:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62841:x1855007616835584:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10785us (12026us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.320859:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14056 00000100:00100000:0.0:1769074772.321037:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836224 00000100:00100000:0.0:1769074772.321044:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836224 00002000:00100000:0.0:1769074772.321084:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3891199) req@ffff89b5497aad80 x1855007616836224/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.321095:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aad80 with x1855007616836224 ext(3145728->3891199) 00010000:00010000:0.0:1769074772.321102:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 11 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.321127:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 11 pid: 28954 timeout: 1860 lvb_type: 0 00002000:00010000:0.0:1769074772.321135:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497aad80 00000100:00100000:0.0:1769074772.321159:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 763 00000100:00100000:0.0:1769074772.321168:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616836224:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074772.332553:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616836096:12345-192.168.204.60@tcp:4:cat.0 Request processed in 20578us (23564us total) trans 21474841196 rc 0/0 00000100:00100000:3.0:1769074772.332571:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 762 00002000:00100000:3.0:1769074772.332582:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1810431) req@ffff89b549203480 x1855007616836096/t21474841196(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.332591:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549203480 with x1855007616836096 ext(1048576->1810431) 00010000:00010000:3.0:1769074772.332599:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 21 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.332613:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 21 pid: 8396 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.332619:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549203480 00000100:00100000:1.0:1769074772.337394:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.337481:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836352 00000100:00100000:3.0:1769074772.337491:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836352 00000100:00100000:3.0:1769074772.337552:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14057 00000100:00100000:3.0:1769074772.337570:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62852:x1855007616836352:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074772.337589:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.337614:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bbf9 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.337645:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.337655:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 21 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.337663:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.337665:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.337669:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [471040->716799] (req 471040->716799) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 16 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.337706:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 21 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.337717:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 21 pid: 8396 timeout: 1860 lvb_type: 1 00000100:00100000:3.0:1769074772.337730:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625137280:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.337856:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.337867:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 16 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.337876:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625137408:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074772.337899:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.339137:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616836224:12345-192.168.204.60@tcp:4:cat.0 Request processed in 17959us (18865us total) trans 17179874509 rc 0/0 00000100:00100000:2.0:1769074772.339150:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 763 00002000:00100000:2.0:1769074772.339163:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3891199) req@ffff89b5497aad80 x1855007616836224/t17179874509(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.339173:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aad80 with x1855007616836224 ext(3145728->3891199) 00010000:00010000:2.0:1769074772.339182:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 15 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.339197:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 15 pid: 28954 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.339204:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5497aad80 00000100:00100000:3.0:1769074772.348674:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625137280:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074772.349989:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625137408:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074772.350026:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [471040->716799] (req 471040->716799) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.350046:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:2.0:1769074772.350105:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62852:x1855007616836352:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12544us (13543us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.350114:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14057 00000100:00100000:1.0:1769074772.351542:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.351775:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836480 00000100:00100000:2.0:1769074772.351787:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836480 00010000:00010000:2.0:1769074772.351871:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 22 pid: 8396 timeout: 1860 lvb_type: 1 00000100:00100000:2.0:1769074772.351911:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7016 00000100:00100000:2.0:1769074772.351920:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b682062450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616836480:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074772.351935:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074772.351957:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 22 pid: 8396 timeout: 1860 lvb_type: 1 00010000:00010000:2.0:1769074772.351968:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 22 pid: 8396 timeout: 1860 lvb_type: 1 00010000:00010000:2.0:1769074772.351994:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd538fd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bae8 expref: 22 pid: 8396 timeout: 1860 lvb_type: 1 00010000:00010000:2.0:1769074772.352010:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074772.352108:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b682062450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616836480:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 188us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.352116:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7016 00000100:00100000:0.0:1769074772.358033:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.358139:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836608 00000100:00100000:1.0:1769074772.358151:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836608 00010000:00010000:1.0:1769074772.358191:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.358240:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7017 00000100:00100000:1.0:1769074772.358250:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616836608:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074772.358266:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074772.358288:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.358300:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.358333:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867200/0x7fe2323adfd539f9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->3891199) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbba expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.358352:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.358354:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.358364:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.358376:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.358428:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.358445:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625137536:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074772.358546:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.360414:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.360491:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836736 00000100:00100000:2.0:1769074772.360501:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836736 00010000:00010000:2.0:1769074772.360543:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 21 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:2.0:1769074772.360581:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7018 00000100:00100000:2.0:1769074772.360591:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616836736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074772.360605:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074772.360623:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 21 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.360637:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 21 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.360659:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd539cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1810431) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb90 expref: 21 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.360673:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.360675:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.360683:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.360693:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.360729:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 12 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074772.360751:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625137664:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074772.360890:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.363706:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.363867:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836864 00000100:00100000:0.0:1769074772.363877:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836864 00010000:00010000:0.0:1769074772.363916:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:0.0:1769074772.363960:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7019 00000100:00100000:0.0:1769074772.363971:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616836864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.363986:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074772.364009:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.364023:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.364047:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd538a2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bad3 expref: 14 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.364065:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.364068:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.364075:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.364084:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.364123:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074772.364137:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625137792:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074772.364245:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.367202:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625137792:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074772.367223:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.367271:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616836864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3302us (3576us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.367280:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7019 00000100:00100000:2.0:1769074772.367353:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625137664:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074772.367366:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074772.367388:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616836736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6799us (6985us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.367396:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7018 00000100:00100000:3.0:1769074772.367495:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625137536:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074772.367504:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074772.367519:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616836608:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9272us (9500us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.367524:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7017 00000100:00100000:1.0:1769074772.370852:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.371051:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616836992 00000100:00100000:2.0:1769074772.371062:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616836992 00002000:00100000:2.0:1769074772.371112:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (471040->475135) req@ffff89b547ef6a00 x1855007616836992/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.371123:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef6a00 with x1855007616836992 ext(471040->475135) 00002000:00010000:2.0:1769074772.371129:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547ef6a00 00000100:00100000:2.0:1769074772.371160:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8543 00000100:00100000:2.0:1769074772.371169:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616836992:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074772.371349:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616836992:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 181us (554us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074772.371357:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8543 00002000:00100000:2.0:1769074772.371364:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (471040->475135) req@ffff89b547ef6a00 x1855007616836992/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.371372:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef6a00 with x1855007616836992 ext(471040->475135) 00002000:00010000:2.0:1769074772.371375:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b547ef6a00 00000100:00100000:1.0:1769074772.382070:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.382112:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:2.0:1769074772.382322:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074772.382346:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837120 00000100:00100000:2.0:1769074772.382359:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837120 00002000:00100000:2.0:1769074772.382424:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b547ef5f80 x1855007616837120/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.382434:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5f80 with x1855007616837120 ext(1048576->2097151) 00002000:00010000:2.0:1769074772.382442:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef5f80 00000100:00100000:2.0:1769074772.382463:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837248 00000100:00100000:2.0:1769074772.382466:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837248 00002000:00100000:2.0:1769074772.382479:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bc:0x0] (712704->716799) req@ffff89b67a0f5180 x1855007616837248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.382484:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5180 with x1855007616837248 ext(712704->716799) 00002000:00010000:2.0:1769074772.382487:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f5180 00000100:00100000:2.0:1769074772.382506:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8544 00000100:00100000:2.0:1769074772.382512:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616837120:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074772.382668:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8545 00000100:00100000:0.0:1769074772.382675:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616837248:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074772.382792:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616837248:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 118us (680us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074772.382798:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8545 00002000:00100000:0.0:1769074772.382803:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (712704->716799) req@ffff89b67a0f5180 x1855007616837248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.382821:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5180 with x1855007616837248 ext(712704->716799) 00002000:00010000:0.0:1769074772.382824:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f5180 00000100:00100000:1.0:1769074772.399294:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.399373:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.399939:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837504 00000100:00100000:2.0:1769074772.399955:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837504 00002000:00100000:2.0:1769074772.400021:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67a0f5f80 x1855007616837504/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.400035:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5f80 with x1855007616837504 ext(3145728->4194303) 00002000:00010000:2.0:1769074772.400043:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f5f80 00000100:00100000:2.0:1769074772.400081:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837376 00000100:00100000:2.0:1769074772.400085:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837376 00002000:00100000:2.0:1769074772.400102:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67a0f7480 x1855007616837376/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.400111:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7480 with x1855007616837376 ext(3145728->4194303) 00002000:00010000:2.0:1769074772.400118:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f7480 00000100:00100000:2.0:1769074772.400160:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8546 00000100:00100000:2.0:1769074772.400172:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616837504:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074772.400696:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8547 00000100:00100000:0.0:1769074772.400709:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616837376:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074772.405104:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616837120:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 22574us (23031us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074772.405122:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8544 00002000:00100000:3.0:1769074772.405137:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b547ef5f80 x1855007616837120/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.405149:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5f80 with x1855007616837120 ext(1048576->2097151) 00002000:00010000:3.0:1769074772.405155:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef5f80 00000100:00100000:0.0:1769074772.427420:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616837376:12345-192.168.204.60@tcp:3:cat.0 Request processed in 26697us (28029us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074772.427436:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8547 00002000:00100000:0.0:1769074772.427453:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67a0f7480 x1855007616837376/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.427465:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f7480 with x1855007616837376 ext(3145728->4194303) 00002000:00010000:0.0:1769074772.427482:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a0f7480 00000100:00100000:3.0:1769074772.429284:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616837504:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29105us (29996us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074772.429296:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8546 00002000:00100000:3.0:1769074772.429308:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b67a0f5f80 x1855007616837504/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.429331:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5f80 with x1855007616837504 ext(3145728->4194303) 00002000:00010000:3.0:1769074772.429338:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67a0f5f80 00000100:00100000:2.0:1769074772.454776:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.454885:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.454907:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.454933:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837760 00000100:00100000:2.0:1769074772.454943:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837760 00000100:00100000:2.0:1769074772.455000:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837632 00000100:00100000:2.0:1769074772.455004:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837632 00000100:00100000:2.0:1769074772.455020:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616837888 00000100:00100000:2.0:1769074772.455023:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616837888 00000100:00100000:2.0:1769074772.455045:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14058 00000100:00100000:2.0:1769074772.455055:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616837760:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.455078:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.455102:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc07 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.455132:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.455135:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.455146:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.455164:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.455179:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031600, rc 0) 00000100:00100000:2.0:1769074772.455246:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616837760:12345-192.168.204.60@tcp:101:cat.0 Request processed in 191us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.455255:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14058 00000100:00100000:2.0:1769074772.455645:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14059 00000100:00100000:2.0:1769074772.455666:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616837632:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.455677:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.455692:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a62 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc00 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074772.455747:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625137920:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074772.455840:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.455865:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14060 00000100:00100000:2.0:1769074772.455873:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616837888:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.455880:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.455889:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc0e expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.456379:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.456412:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838016 00000100:00100000:3.0:1769074772.456421:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838016 00000100:00100000:3.0:1769074772.456463:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14061 00000100:00100000:3.0:1769074772.456483:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62850:x1855007616838016:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074772.456492:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.456503:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc15 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456522:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 24 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456531:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456537:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.456539:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.456542:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2293760->2887679] (req 2293760->2887679) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456576:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456592:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 24 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.456604:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138048:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.456660:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 24 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456674:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 24 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.456684:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138176:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074772.456704:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.456766:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.456787:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838144 00000100:00100000:3.0:1769074772.456791:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838144 00000100:00100000:3.0:1769074772.456844:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14062 00000100:00100000:3.0:1769074772.456851:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616838144:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.456859:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.456868:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd53a77 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc1c expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.456881:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.456893:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd53a77 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc1c expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.456909:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ce00/0x7fe2323adfd53a77 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc1c expref: 13 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.456922:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:3.0:1769074772.456948:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616838144:12345-192.168.204.60@tcp:101:cat.0 Request processed in 98us (182us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074772.456955:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14062 00000100:00100000:3.0:1769074772.457008:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.457023:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838272 00000100:00100000:3.0:1769074772.457027:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838272 00000100:00100000:3.0:1769074772.457047:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14063 00000100:00100000:3.0:1769074772.457053:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616838272:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.457060:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.457070:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc23 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.457083:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.457085:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.457093:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.457106:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.457115:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:3.0:1769074772.457135:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616838272:12345-192.168.204.60@tcp:101:cat.0 Request processed in 82us (127us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.457140:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14063 00010000:00010000:2.0:1769074772.457826:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.457831:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.457838:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 22 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.457854:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 22 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.457865:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030800, rc 0) 00000100:00100000:2.0:1769074772.457911:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616837888:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2039us (3003us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.457919:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14060 00000100:00100000:0.0:1769074772.463922:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.463954:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838400 00000100:00100000:0.0:1769074772.463965:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838400 00010000:00010000:0.0:1769074772.464000:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 25 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:0.0:1769074772.464031:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7020 00000100:00100000:0.0:1769074772.464037:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616838400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.464050:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074772.464067:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 25 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.464079:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 25 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.464106:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.464160:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616838400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 123us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.464166:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7020 00000100:00100000:0.0:1769074772.464249:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.464301:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838528 00000100:00100000:0.0:1769074772.464304:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838528 00010000:00010000:0.0:1769074772.464324:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 25 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:0.0:1769074772.464344:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7021 00000100:00100000:0.0:1769074772.464351:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616838528:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.464363:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074772.464371:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 25 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.464381:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 25 pid: 28954 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.464393:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.464397:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.464404:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.464413:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.464450:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074772.464701:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625138304:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074772.464776:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499e6900 going to sleep for 11 seconds 00002000:00010000:2.0:1769074772.465121:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime: 1769074770 -> 1769074772 00002000:00010000:2.0:1769074772.465125:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime: 1769074770 -> 1769074772 00002000:00010000:2.0:1769074772.465127:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime: 1769074770 -> 1769074772 00000100:00100000:2.0:1769074772.465139:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625137920:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074772.465165:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a62 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc00 expref: 25 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.465179:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a62 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc00 expref: 25 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.465190:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030400, rc 0) 00000100:00100000:2.0:1769074772.465220:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616837632:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9567us (10334us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074772.465227:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14059 00010000:00010000:3.0:1769074772.465394:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d000/0x7fe2323adfd53a4d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf2 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.465405:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138048:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.465416:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd539f2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->3145727] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbb3 expref: 22 pid: 28954 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.465431:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138176:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.465453:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.465462:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c600, rc 0) 00000100:00100000:3.0:1769074772.465497:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62850:x1855007616838016:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 9027us (9120us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.465503:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14061 00000100:00100000:0.0:1769074772.470394:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.470558:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838656 00000100:00100000:3.0:1769074772.470567:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838656 00000100:00100000:3.0:1769074772.470650:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14064 00000100:00100000:3.0:1769074772.470669:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62847:x1855007616838656:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.470691:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.470718:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc2a expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.470755:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.470765:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.470767:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074772.470771:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.471008:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.471024:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 20 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.471039:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138432:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074772.471247:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.493142:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.493320:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625138304:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074772.493354:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074772.493432:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616838528:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 29079us (29179us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.493453:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7021 00000100:00100000:2.0:1769074772.493481:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838784 00000100:00100000:2.0:1769074772.493490:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838784 00000100:00100000:2.0:1769074772.493546:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14065 00000100:00100000:2.0:1769074772.493555:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62841:x1855007616838784:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.493574:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.493595:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc31 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.493629:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.493632:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.493637:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.493652:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.493672:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676030400, rc 0) 00000100:00100000:2.0:1769074772.493712:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62841:x1855007616838784:12345-192.168.204.60@tcp:101:cat.0 Request processed in 157us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.493717:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14065 00000100:00100000:3.0:1769074772.493981:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138432:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.494006:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 13 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.494016:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:3.0:1769074772.494041:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62847:x1855007616838656:12345-192.168.204.60@tcp:101:cat.0 Request processed in 23384us (23659us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.494049:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14064 00000100:00100000:0.0:1769074772.494943:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.495922:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616838912 00000100:00100000:3.0:1769074772.495935:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616838912 00002000:00100000:3.0:1769074772.495986:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2293760->2297855) req@ffff89b5461e5f80 x1855007616838912/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.495996:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5f80 with x1855007616838912 ext(2293760->2297855) 00010000:00010000:3.0:1769074772.496005:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.496021:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.496029:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5461e5f80 00000100:00100000:3.0:1769074772.496067:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 764 00000100:00100000:3.0:1769074772.496077:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616838912:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074772.496301:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616838912:12345-192.168.204.60@tcp:3:cat.0 Request processed in 224us (1363us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074772.496319:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 764 00002000:00100000:3.0:1769074772.496327:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2293760->2297855) req@ffff89b5461e5f80 x1855007616838912/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.496335:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5f80 with x1855007616838912 ext(2293760->2297855) 00010000:00010000:3.0:1769074772.496341:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.496351:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.496358:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5461e5f80 00000100:00100000:0.0:1769074772.521065:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.522721:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839040 00000100:00100000:2.0:1769074772.522746:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839040 00002000:00100000:2.0:1769074772.522873:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2883584->2887679) req@ffff89b5461e7800 x1855007616839040/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.522889:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7800 with x1855007616839040 ext(2883584->2887679) 00010000:00010000:2.0:1769074772.522902:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.522922:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.522930:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5461e7800 00000100:00100000:2.0:1769074772.522974:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 765 00000100:00100000:2.0:1769074772.522985:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616839040:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074772.523232:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616839040:12345-192.168.204.60@tcp:3:cat.0 Request processed in 249us (2190us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074772.523238:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 765 00002000:00100000:2.0:1769074772.523244:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2883584->2887679) req@ffff89b5461e7800 x1855007616839040/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.523250:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7800 with x1855007616839040 ext(2883584->2887679) 00010000:00010000:2.0:1769074772.523253:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.523262:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.523266:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5461e7800 00000100:00100000:1.0:1769074772.530151:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.530319:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839168 00000100:00100000:2.0:1769074772.530333:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839168 00002000:00100000:2.0:1769074772.530402:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2293760->2887679) req@ffff89b67a0f5c00 x1855007616839168/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.530416:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5c00 with x1855007616839168 ext(2293760->2887679) 00010000:00010000:2.0:1769074772.530429:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.530446:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 21 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.530454:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a0f5c00 00000100:00100000:2.0:1769074772.531777:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 766 00000100:00100000:2.0:1769074772.531801:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616839168:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074772.546940:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616839168:12345-192.168.204.60@tcp:4:cat.0 Request processed in 15137us (16802us total) trans 17179874510 rc 0/0 00000100:00100000:3.0:1769074772.546954:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 766 00002000:00100000:3.0:1769074772.546968:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2293760->2887679) req@ffff89b67a0f5c00 x1855007616839168/t17179874510(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.546979:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5c00 with x1855007616839168 ext(2293760->2887679) 00010000:00010000:3.0:1769074772.546989:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.547010:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.547020:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a0f5c00 00000100:00100000:0.0:1769074772.553352:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.553468:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839296 00000100:00100000:1.0:1769074772.553483:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839296 00010000:00010000:1.0:1769074772.553530:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.553580:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7022 00000100:00100000:1.0:1769074772.553591:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616839296:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074772.553609:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074772.553631:0:36300:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074771 -> 1769074772 00002000:00010000:1.0:1769074772.553634:0:36300:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074771 -> 1769074772 00010000:00010000:1.0:1769074772.553642:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.553656:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.553685:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd53a70 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2293760->2887679) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc15 expref: 23 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.553703:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.553705:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.553712:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.553722:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.553732:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.553733:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.553737:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.553747:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.553791:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.553895:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625138560:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074772.554330:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 12 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.554348:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625138688:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074772.554377:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074772.570843:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625138560:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074772.570865:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f97800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625138688:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074772.570893:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074772.570965:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616839296:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17375us (17628us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.570979:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7022 00000100:00100000:1.0:1769074772.583170:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.583306:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839424 00000100:00100000:2.0:1769074772.583325:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839424 00002000:00100000:2.0:1769074772.583391:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f97800 x1855007616839424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.583404:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97800 with x1855007616839424 ext(4194304->4808703) 00002000:00010000:2.0:1769074772.583411:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f97800 00000100:00100000:2.0:1769074772.583461:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8548 00000100:00100000:2.0:1769074772.583472:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616839424:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074772.595063:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.596940:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839552 00000100:00100000:3.0:1769074772.596958:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839552 00002000:00100000:2.0:1769074772.598892:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5446c0e00 x1855007616839552/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.598917:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0e00 with x1855007616839552 ext(2097152->4194303) 00002000:00010000:2.0:1769074772.598924:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c0e00 00000100:00100000:2.0:1769074772.598967:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8549 00000100:00100000:2.0:1769074772.598981:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616839552:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074772.604356:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.604456:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839680 00000100:00100000:0.0:1769074772.604480:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839680 00002000:00100000:0.0:1769074772.604535:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546f97100 x1855007616839680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.604553:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97100 with x1855007616839680 ext(2097152->4194303) 00002000:00010000:0.0:1769074772.604559:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f97100 00000100:00100000:0.0:1769074772.604585:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8550 00000100:00100000:0.0:1769074772.604595:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616839680:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074772.607749:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616839424:12345-192.168.204.60@tcp:3:cat.0 Request processed in 24268us (24583us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074772.607766:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8548 00002000:00100000:2.0:1769074772.607782:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b546f97800 x1855007616839424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.607792:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97800 with x1855007616839424 ext(4194304->4808703) 00002000:00010000:2.0:1769074772.607883:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b546f97800 00000100:00100000:2.0:1769074772.654755:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616839552:12345-192.168.204.60@tcp:3:cat.0 Request processed in 55765us (59689us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074772.654772:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8549 00002000:00100000:2.0:1769074772.654794:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5446c0e00 x1855007616839552/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.654834:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0e00 with x1855007616839552 ext(2097152->4194303) 00002000:00010000:2.0:1769074772.654842:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c0e00 00000100:00100000:0.0:1769074772.659743:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.660090:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839808 00000100:00100000:3.0:1769074772.660102:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839808 00000100:00100000:3.0:1769074772.660172:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14066 00000100:00100000:3.0:1769074772.660181:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616839808:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074772.660199:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.660216:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd53a93 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc38 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.660273:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138816:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074772.660378:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074772.661492:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616839680:12345-192.168.204.60@tcp:3:cat.0 Request processed in 56890us (57141us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074772.661505:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8550 00002000:00100000:3.0:1769074772.661515:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b546f97100 x1855007616839680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.661525:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f97100 with x1855007616839680 ext(2097152->4194303) 00002000:00010000:3.0:1769074772.661531:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b546f97100 00000100:00100000:3.0:1769074772.664950:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625138816:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074772.665009:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd53a93 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc38 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.665041:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c000/0x7fe2323adfd53a93 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc38 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.665052:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c000, rc 0) 00000100:00100000:3.0:1769074772.665242:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616839808:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5060us (5507us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074772.665252:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14066 00000100:00100000:0.0:1769074772.666868:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.666959:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616839936 00000100:00100000:3.0:1769074772.666968:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616839936 00000100:00100000:3.0:1769074772.667027:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14067 00000100:00100000:3.0:1769074772.667036:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62856:x1855007616839936:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074772.667056:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.667076:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc3f expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.667112:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.667122:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 15 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.667138:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.667140:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.667144:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2400256->2416639] (req 2400256->2416639) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.667187:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 15 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074772.667206:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 15 pid: 27844 timeout: 1860 lvb_type: 1 00000100:00100000:3.0:1769074772.667221:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625138944:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.667722:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.667733:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 21 pid: 28955 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.667742:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625139072:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074772.667771:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.679317:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.679393:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.679432:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.679467:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840320 00000100:00100000:0.0:1769074772.679476:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840320 00010000:00010000:0.0:1769074772.679513:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 16 pid: 27844 timeout: 1860 lvb_type: 1 00000100:00100000:0.0:1769074772.679550:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7023 00000100:00100000:0.0:1769074772.679560:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b682061450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616840320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.679578:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074772.679602:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 16 pid: 27844 timeout: 1860 lvb_type: 1 00010000:00010000:0.0:1769074772.679614:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 16 pid: 27844 timeout: 1860 lvb_type: 1 00010000:00010000:0.0:1769074772.679641:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.679701:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b682061450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616840320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 141us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.679709:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7023 00000100:00100000:2.0:1769074772.679982:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840064 00000100:00100000:2.0:1769074772.679988:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840064 00000100:00100000:2.0:1769074772.680021:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840192 00000100:00100000:2.0:1769074772.680024:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840192 00000100:00100000:2.0:1769074772.680045:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14068 00000100:00100000:2.0:1769074772.680052:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616840064:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.680068:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.680085:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd53aa1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc46 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074772.680138:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625139200:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074772.680195:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.680213:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14069 00000100:00100000:2.0:1769074772.680221:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62847:x1855007616840192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074772.680228:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.680236:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc4d expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.680250:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.680256:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2400256->2416639] (req 2400256->2416639) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.680263:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074772.680268:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.680295:0:8396:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2400256->2416639] (req 2400256->2416639) gid 0 flags: 0x60000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.680319:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.680330:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031e00, rc 0) 00000100:00100000:2.0:1769074772.680352:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62847:x1855007616840192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 131us (959us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074772.680358:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14069 00010000:00010000:3.0:1769074772.680575:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53a7e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc23 expref: 16 pid: 27844 timeout: 1860 lvb_type: 1 00000100:00100000:3.0:1769074772.680590:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625138944:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074772.680602:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625139072:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074772.680622:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2400256->2416639] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.680636:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d000, rc 0) 00000100:00100000:3.0:1769074772.680674:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62856:x1855007616839936:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 13640us (13816us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.680681:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14067 00000100:00100000:0.0:1769074772.683571:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.683980:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840448 00000100:00100000:2.0:1769074772.683991:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840448 00010000:00010000:2.0:1769074772.684030:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 22 pid: 28955 timeout: 1860 lvb_type: 0 00000100:00100000:2.0:1769074772.684069:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7024 00000100:00100000:2.0:1769074772.684079:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616840448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074772.684094:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074772.684115:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 22 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.684132:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 22 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.684161:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d200/0x7fe2323adfd53a15 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbd6 expref: 22 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.684177:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.684179:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.684186:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.684197:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.684239:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.684252:0:6499:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:2.0:1769074772.684266:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625139456:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074772.684370:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.685408:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625139200:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074772.685445:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd53aa1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc46 expref: 23 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074772.685464:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031400/0x7fe2323adfd53aa1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc46 expref: 23 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074772.685475:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b676031400, rc 0) 00000100:00100000:0.0:1769074772.685574:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616840064:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5521us (6285us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074772.685584:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14068 00000100:00100000:2.0:1769074772.692334:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.692496:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840576 00000100:00100000:3.0:1769074772.692507:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840576 00002000:00100000:3.0:1769074772.692557:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2400256->2404351) req@ffff89b67ff0ca80 x1855007616840576/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.692567:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ca80 with x1855007616840576 ext(2400256->2404351) 00010000:00010000:3.0:1769074772.692576:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.692590:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.692597:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67ff0ca80 00000100:00100000:3.0:1769074772.692661:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 767 00000100:00100000:3.0:1769074772.692670:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616840576:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074772.692855:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616840576:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 185us (530us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074772.692863:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 767 00002000:00100000:3.0:1769074772.692870:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2400256->2404351) req@ffff89b67ff0ca80 x1855007616840576/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.692884:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ca80 with x1855007616840576 ext(2400256->2404351) 00010000:00010000:3.0:1769074772.692889:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.692899:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.692905:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67ff0ca80 00000100:00100000:0.0:1769074772.694908:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625139456:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074772.694942:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.695029:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616840448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10950us (11468us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.695041:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7024 00000100:00100000:1.0:1769074772.704669:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.704838:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840704 00000100:00100000:3.0:1769074772.704849:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840704 00002000:00100000:3.0:1769074772.704907:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2412544->2416639) req@ffff89b67a0f5880 x1855007616840704/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.704918:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5880 with x1855007616840704 ext(2412544->2416639) 00010000:00010000:3.0:1769074772.704927:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.704943:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.704950:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a0f5880 00000100:00100000:3.0:1769074772.704988:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 768 00000100:00100000:3.0:1769074772.704998:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616840704:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074772.705177:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616840704:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 180us (531us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074772.705188:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 768 00002000:00100000:3.0:1769074772.705195:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (2412544->2416639) req@ffff89b67a0f5880 x1855007616840704/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.705203:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a0f5880 with x1855007616840704 ext(2412544->2416639) 00010000:00010000:3.0:1769074772.705207:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.705217:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:3.0:1769074772.705224:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a0f5880 00000100:00100000:1.0:1769074772.708609:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.709209:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616840960 00000100:00100000:3.0:1769074772.709221:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616840960 00000100:00100000:3.0:1769074772.709287:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13318 00000100:00100000:3.0:1769074772.709297:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62858:x1855007616840960:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074772.709338:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074772.709361:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53aaf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc54 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.709440:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709456:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709467:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709484:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709490:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709495:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709615:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53ac4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709625:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53ac4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709632:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53ac4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074772.709642:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074772.709741:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709749:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709754:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709760:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709765:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074772.709769:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709776:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709788:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074772.709791:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074772.709796:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709802:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ab6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709847:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709852:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709857:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709862:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910ce00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074772.709865:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709871:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709877:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6439ad500) 00010000:00010000:3.0:1769074772.709880:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074772.709883:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709888:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53abd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074772.709929:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53aaf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc54 expref: 250 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.709942:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53ac4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc54 expref: 249 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.709949:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074772.709951:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074772.709953:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910da00, rc 0) 00000100:00100000:3.0:1769074772.710013:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62858:x1855007616840960:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 718us (1415us total) trans 8589943943 rc 0/0 00000100:00100000:3.0:1769074772.710021:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13318 00000100:00100000:1.0:1769074772.712746:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.713549:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841088 00000100:00100000:2.0:1769074772.713563:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841088 00002000:00100000:2.0:1769074772.713632:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2400256->2416639) req@ffff89b54d101c00 x1855007616841088/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 16872/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.713645:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101c00 with x1855007616841088 ext(2400256->2416639) 00010000:00010000:2.0:1769074772.713659:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.713678:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.713687:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d101c00 00000100:00100000:2.0:1769074772.713717:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 769 00000100:00100000:2.0:1769074772.713728:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616841088:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074772.715035:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616841088:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 1297us (2291us total) trans 21474841197 rc 0/0 00000100:00100000:2.0:1769074772.715050:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 769 00002000:00100000:2.0:1769074772.715060:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2400256->2416639) req@ffff89b54d101c00 x1855007616841088/t21474841197(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 16872/448 e 0 to 0 dl 1769074783 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.715071:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d101c00 with x1855007616841088 ext(2400256->2416639) 00010000:00010000:2.0:1769074772.715083:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074772.715102:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074772.715109:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d101c00 00000100:00100000:1.0:1769074772.720022:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.720342:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.720791:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841472 00000100:00100000:0.0:1769074772.720802:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841472 00000100:00100000:0.0:1769074772.720899:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1008 00000100:00100000:0.0:1769074772.720911:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:2409:x1855007616841472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.720934:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074772.720961:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910da00/0x7fe2323adfd53ac4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc54 expref: 249 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.720988:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074772.720993:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.721061:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:2409:x1855007616841472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 152us (719us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.721070:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1008 00000100:00100000:3.0:1769074772.722217:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841344 00000100:00100000:3.0:1769074772.722230:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841344 00000100:00100000:2.0:1769074772.722575:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4666 00000100:00100000:2.0:1769074772.722587:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62858:x1855007616841344:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074772.723232:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62858:x1855007616841344:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 644us (3219us total) trans 8589943944 rc 0/0 00000100:00100000:2.0:1769074772.723241:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4666 00000100:00100000:1.0:1769074772.727738:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.727877:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841600 00000100:00100000:0.0:1769074772.727889:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841600 00010000:00010000:0.0:1769074772.727930:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:0.0:1769074772.728018:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7025 00000100:00100000:0.0:1769074772.728029:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616841600:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074772.728049:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1769074772.728072:0:36300:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074771 -> 1769074772 00002000:00010000:0.0:1769074772.728075:0:36300:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074771 -> 1769074772 00010000:00010000:0.0:1769074772.728082:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.728097:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.728135:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910d000/0x7fe2323adfd53a9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2400256->2416639) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc3f expref: 17 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074772.728156:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.728159:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074772.728167:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.728176:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.728215:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074772.728229:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625139584:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074772.728501:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.731659:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625139584:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074772.731693:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074772.731761:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616841600:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3733us (4112us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.731773:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7025 00000100:00100000:1.0:1769074772.743047:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.743156:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841728 00000100:00100000:2.0:1769074772.743167:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841728 00002000:00100000:2.0:1769074772.743214:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54cfef100 x1855007616841728/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.743226:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfef100 with x1855007616841728 ext(2097152->4194303) 00002000:00010000:2.0:1769074772.743231:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfef100 00000100:00100000:2.0:1769074772.743321:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8551 00000100:00100000:2.0:1769074772.743330:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616841728:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074772.758342:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.758525:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841856 00000100:00100000:2.0:1769074772.758536:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841856 00000100:00100000:2.0:1769074772.758597:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14070 00000100:00100000:2.0:1769074772.758608:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62858:x1855007616841856:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074772.758628:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074772.758657:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc5b expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.758697:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.758707:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.758714:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.758716:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074772.758721:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1515520->1916927] (req 1515520->1916927) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.758756:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.758769:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 1860 lvb_type: 1 00000100:00100000:2.0:1769074772.758783:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625139712:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074772.758907:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 17 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074772.758916:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 17 pid: 28955 timeout: 1860 lvb_type: 0 00000100:00100000:2.0:1769074772.758925:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625139840:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074772.758949:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.765895:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625139712:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074772.765912:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625139840:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074772.765946:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1515520->1916927] (req 1515520->1916927) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 22 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074772.765965:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446800, rc 0) 00000100:00100000:0.0:1769074772.766032:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62858:x1855007616841856:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 7425us (7706us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074772.766042:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14070 00000100:00100000:1.0:1769074772.767281:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.767331:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.767682:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616841984 00000100:00100000:3.0:1769074772.767694:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616841984 00010000:00010000:3.0:1769074772.767748:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 1860 lvb_type: 1 00000100:00100000:3.0:1769074772.767797:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842112 00000100:00100000:3.0:1769074772.767801:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842112 00010000:00010000:3.0:1769074772.767858:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 18 pid: 28955 timeout: 1860 lvb_type: 0 00000100:00100000:3.0:1769074772.767891:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7026 00000100:00100000:3.0:1769074772.767902:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616841984:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074772.767921:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074772.767947:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 1860 lvb_type: 1 00010000:00010000:3.0:1769074772.767961:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 1860 lvb_type: 1 00010000:00010000:3.0:1769074772.767984:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b676031600/0x7fe2323adfd53a5b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc07 expref: 22 pid: 28955 timeout: 1860 lvb_type: 1 00010000:00010000:3.0:1769074772.768005:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074772.768064:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616841984:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 164us (798us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074772.768071:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7026 00000100:00100000:3.0:1769074772.768090:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7027 00000100:00100000:3.0:1769074772.768096:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616842112:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074772.768105:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074772.768112:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 18 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.768123:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 18 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.768136:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53a1c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbdd expref: 18 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:3.0:1769074772.768146:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.768148:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074772.768157:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.768170:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074772.768207:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074772.768221:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625139968:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074772.768285:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c510c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074772.771728:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625139968:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074772.771757:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074772.771843:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616842112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3745us (4511us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.771856:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7027 00000100:00100000:3.0:1769074772.772518:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.772599:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842240 00000100:00100000:3.0:1769074772.772607:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842240 00002000:00100000:3.0:1769074772.772658:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1515520->1519615) req@ffff89b54385ad80 x1855007616842240/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.772669:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385ad80 with x1855007616842240 ext(1515520->1519615) 00002000:00010000:3.0:1769074772.772674:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385ad80 00000100:00100000:3.0:1769074772.772700:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8552 00000100:00100000:3.0:1769074772.772708:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616842240:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074772.772880:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616842240:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 171us (363us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074772.772887:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8552 00002000:00100000:3.0:1769074772.772893:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1515520->1519615) req@ffff89b54385ad80 x1855007616842240/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074772.772907:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385ad80 with x1855007616842240 ext(1515520->1519615) 00002000:00010000:3.0:1769074772.772910:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385ad80 00000100:00100000:2.0:1769074772.775070:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616841728:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 31726us (32019us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074772.775083:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8551 00002000:00100000:2.0:1769074772.775095:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54cfef100 x1855007616841728/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074772.775105:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfef100 with x1855007616841728 ext(2097152->4194303) 00002000:00010000:2.0:1769074772.775111:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfef100 00000100:00100000:3.0:1769074772.792733:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.793469:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842368 00000100:00100000:1.0:1769074772.793480:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842368 00002000:00100000:1.0:1769074772.793534:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1912832->1916927) req@ffff89b543859880 x1855007616842368/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074772.793546:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859880 with x1855007616842368 ext(1912832->1916927) 00002000:00010000:1.0:1769074772.793552:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543859880 00000100:00100000:1.0:1769074772.793585:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8553 00000100:00100000:1.0:1769074772.793594:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616842368:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074772.793798:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616842368:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 204us (1077us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074772.793845:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8553 00002000:00100000:1.0:1769074772.793853:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2ba:0x0] (1912832->1916927) req@ffff89b543859880 x1855007616842368/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074772.793861:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859880 with x1855007616842368 ext(1912832->1916927) 00002000:00010000:1.0:1769074772.793865:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543859880 00000100:00100000:1.0:1769074772.795681:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.796009:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842496 00000100:00100000:0.0:1769074772.796018:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842496 00000100:00100000:0.0:1769074772.796072:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14071 00000100:00100000:0.0:1769074772.796082:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616842496:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074772.796101:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074772.796132:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd53ad9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc62 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074772.796197:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625140096:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074772.796278:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00002000:00010000:0.0:1769074772.801735:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074771 -> 1769074772 00002000:00010000:0.0:1769074772.801749:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074771 -> 1769074772 00002000:00010000:0.0:1769074772.801752:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074771 -> 1769074772 00000100:00100000:0.0:1769074772.801777:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625140096:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074772.801935:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd53ad9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc62 expref: 16 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074772.801959:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd53ad9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc62 expref: 16 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074772.802039:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877800, rc 0) 00000100:00100000:1.0:1769074772.802327:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616842496:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6245us (6653us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074772.802337:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14071 00000100:00100000:3.0:1769074772.807112:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074772.812793:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842624 00000100:00100000:0.0:1769074772.812842:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842624 00002000:00100000:0.0:1769074772.813070:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b543859c00 x1855007616842624/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.813087:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859c00 with x1855007616842624 ext(4194304->4808703) 00002000:00010000:0.0:1769074772.813097:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543859c00 00000100:00100000:0.0:1769074772.813256:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8554 00000100:00100000:0.0:1769074772.813268:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616842624:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074772.834752:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616842624:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21474us (27644us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074772.834769:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8554 00002000:00100000:0.0:1769074772.834782:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b543859c00 x1855007616842624/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/440 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.834792:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859c00 with x1855007616842624 ext(4194304->4808703) 00002000:00010000:0.0:1769074772.834797:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543859c00 00000100:00100000:2.0:1769074772.850178:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.860716:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842752 00000100:00100000:1.0:1769074772.860733:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842752 00000100:00100000:1.0:1769074772.860797:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14072 00000100:00100000:1.0:1769074772.860835:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616842752:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.860855:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.860875:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53ae0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc69 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.860937:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625140224:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074772.861037:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074772.866797:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625140224:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074772.866888:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53ae0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc69 expref: 16 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074772.866911:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53ae0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc69 expref: 16 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074772.866924:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea0400, rc 0) 00000100:00100000:1.0:1769074772.867008:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616842752:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6201us (16840us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074772.867017:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14072 00000100:00100000:2.0:1769074772.881968:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.882363:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616842880 00000100:00100000:1.0:1769074772.882380:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616842880 00000100:00100000:1.0:1769074772.882458:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14073 00000100:00100000:1.0:1769074772.882477:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616842880:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.882496:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.882526:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53ae7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc70 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.882594:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625140352:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074772.882705:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074772.908771:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.909003:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616843008 00000100:00100000:1.0:1769074772.909016:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616843008 00000100:00100000:1.0:1769074772.909085:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14074 00000100:00100000:1.0:1769074772.909096:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62863:x1855007616843008:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074772.909124:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.909151:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc77 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.909190:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.909201:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.909204:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.909208:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2822144->3145727] (req 2822144->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.909257:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.909277:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 18 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.909313:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625140480:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074772.909388:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.909491:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625140352:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074772.909524:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53ae7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc70 expref: 18 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.909544:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53ae7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc70 expref: 18 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074772.909556:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea0400, rc 0) 00000100:00100000:2.0:1769074772.909589:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616842880:12345-192.168.204.60@tcp:101:cat.0 Request processed in 27122us (27632us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074772.909597:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14073 00000100:00100000:1.0:1769074772.943886:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625140480:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074772.944275:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2822144->3145727] (req 2822144->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.944314:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea1400, rc 0) 00000100:00100000:1.0:1769074772.944393:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62863:x1855007616843008:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 35297us (35633us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.944405:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14074 00000100:00100000:2.0:1769074772.965580:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.966623:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616843136 00000100:00100000:1.0:1769074772.966637:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616843136 00010000:00010000:1.0:1769074772.966696:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074772.966748:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7028 00000100:00100000:1.0:1769074772.966758:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2408:x1855007616843136:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074772.967595:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074772.967625:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.967642:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.967678:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b676031e00/0x7fe2323adfd53aa8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc4d expref: 16 pid: 8396 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074772.967696:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.967698:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074772.967704:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.967719:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.967765:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074772.968126:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625140608:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074772.969026:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1769074772.985416:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074772.985856:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616843392 00000100:00100000:1.0:1769074772.985869:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616843392 00000100:00100000:1.0:1769074772.985943:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13319 00000100:00100000:1.0:1769074772.985954:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62867:x1855007616843392:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074772.985979:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074772.986003:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53af5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc7e expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.986085:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986098:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986109:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986123:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986135:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986141:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986259:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b0a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986271:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b0a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986278:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b0a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074772.986289:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074772.986377:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986385:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986391:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986398:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986404:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641650800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074772.986409:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986416:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986431:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074772.986434:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074772.986439:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986444:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd53afc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986469:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986475:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986481:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986487:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641651e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074772.986490:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986495:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986503:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b654294000) 00010000:00010000:1.0:1769074772.986505:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074772.986508:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986514:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b03 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074772.986554:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0400/0x7fe2323adfd53af5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc7e expref: 252 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.986568:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b0a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc7e expref: 251 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.986575:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074772.986577:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074772.986580:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:1.0:1769074772.986664:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62867:x1855007616843392:12345-192.168.204.60@tcp:101:cat.0 Request processed in 711us (1260us total) trans 8589943945 rc 0/0 00000100:00100000:1.0:1769074772.986673:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13319 00000100:00100000:3.0:1769074772.991183:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.994644:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074772.994692:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074772.995200:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616843904 00000100:00100000:2.0:1769074772.995211:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616843904 00000100:00100000:2.0:1769074772.995263:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4667 00000100:00100000:2.0:1769074772.995273:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c4450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62867:x1855007616843904:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:2.0:1769074772.995514:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c4450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+252:62867:x1855007616843904:12345-192.168.204.60@tcp:35:cat.0 Request processed in 243us (879us total) trans 8589943946 rc 0/0 00000100:00100000:2.0:1769074772.995522:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4667 00000100:00100000:1.0:1769074772.996557:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625140608:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074772.996587:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074772.996637:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616843136:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 29880us (31073us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.996647:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7028 00000100:00100000:1.0:1769074772.996687:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616843776 00000100:00100000:1.0:1769074772.996693:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616843776 00000100:00100000:1.0:1769074772.996736:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1009 00000100:00100000:1.0:1769074772.996743:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b643b0c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+252:2407:x1855007616843776:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074772.996754:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074772.996778:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b0a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc7e expref: 252 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074772.996798:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074772.996828:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074772.996850:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b643b0c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:2407:x1855007616843776:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 107us (2158us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074772.996857:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1009 00000100:00100000:0.0:1769074772.998318:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616843520 00000100:00100000:0.0:1769074772.998331:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616843520 00002000:00100000:0.0:1769074772.998384:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2822144->2826239) req@ffff89b54385a300 x1855007616843520/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/0 e 0 to 0 dl 1769074783 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.998396:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a300 with x1855007616843520 ext(2822144->2826239) 00002000:00010000:0.0:1769074772.998401:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385a300 00000100:00100000:0.0:1769074772.998431:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8555 00000100:00100000:0.0:1769074772.998440:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616843520:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074772.998618:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616843520:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 177us (7444us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074772.998627:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8555 00002000:00100000:0.0:1769074772.998634:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2822144->2826239) req@ffff89b54385a300 x1855007616843520/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:308/0 lens 488/4536 e 0 to 0 dl 1769074783 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074772.998654:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a300 with x1855007616843520 ext(2822144->2826239) 00002000:00010000:0.0:1769074772.998657:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385a300 00000100:00100000:0.0:1769074773.008395:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.008530:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.011517:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844032 00000100:00100000:1.0:1769074773.011534:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844032 00000100:00100000:1.0:1769074773.011617:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844160 00000100:00100000:1.0:1769074773.011621:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844160 00000100:00100000:1.0:1769074773.011655:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14075 00000100:00100000:1.0:1769074773.011664:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616844032:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074773.011683:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074773.011701:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641651a00/0x7fe2323adfd53b18 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc85 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.011762:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625140736:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074773.011852:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.011874:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14076 00000100:00100000:1.0:1769074773.011881:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616844160:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074773.011889:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074773.011898:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b1f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc8c expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.011927:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625140864:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074773.011950:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00080000:0.0:1769074773.012065:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1769074773.020943:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074771 -> 1769074772 00000100:00100000:2.0:1769074773.020977:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625140864:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074773.021029:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b1f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc8c expref: 22 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.021060:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b1f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc8c expref: 22 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.021078:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651e00, rc 0) 00000100:00100000:2.0:1769074773.021152:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616844160:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9268us (12621us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074773.021162:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14076 00000100:00100000:1.0:1769074773.021597:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625140736:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074773.021627:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b641651a00/0x7fe2323adfd53b18 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc85 expref: 23 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074773.021650:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641651a00/0x7fe2323adfd53b18 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc85 expref: 23 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074773.021660:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:1.0:1769074773.021694:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616844032:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10031us (13311us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074773.021701:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14075 00000100:00100000:3.0:1769074773.026574:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.026700:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844288 00000100:00100000:1.0:1769074773.026712:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844288 00000100:00100000:1.0:1769074773.026782:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14077 00000100:00100000:1.0:1769074773.026792:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62863:x1855007616844288:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074773.026824:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074773.026844:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc93 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.026874:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.026884:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.026892:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.026894:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.026898:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3284991] (req 3145728->3284991) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 14 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.026936:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 21 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.026949:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 21 pid: 28955 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074773.026962:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625140992:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074773.027055:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.027064:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074773.027074:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625141120:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074773.027098:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00080000:0.0:1769074773.027170:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074773.028887:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.029372:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844416 00000100:00100000:2.0:1769074773.029390:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844416 00000100:00100000:2.0:1769074773.029482:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14078 00000100:00100000:2.0:1769074773.029495:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62867:x1855007616844416:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074773.029520:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074773.029549:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bc9a expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.029594:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.029628:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074773.029631:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074773.029638:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.029675:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 14 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.029691:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:2.0:1769074773.029704:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625141248:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074773.029884:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.035375:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625140992:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074773.035393:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625141120:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074773.035425:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->3284991] (req 3145728->3284991) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 14 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.035447:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650800, rc 0) 00000100:00100000:1.0:1769074773.035510:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62863:x1855007616844288:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 8718us (8945us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.035523:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14077 00000100:00100000:2.0:1769074773.036110:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625141248:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074773.036148:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 23 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.036165:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9a00, rc 0) 00000100:00100000:2.0:1769074773.036237:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62867:x1855007616844416:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6744us (7357us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.036248:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14078 00000100:00100000:3.0:1769074773.043337:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.043400:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.043605:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844672 00000100:00100000:0.0:1769074773.043617:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844672 00010000:00010000:0.0:1769074773.043667:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:0.0:1769074773.043708:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844544 00000100:00100000:0.0:1769074773.043712:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844544 00010000:00010000:0.0:1769074773.043725:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 23 pid: 28955 timeout: 1860 lvb_type: 0 00000100:00100000:0.0:1769074773.043754:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7029 00000100:00100000:0.0:1769074773.043763:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616844672:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074773.043777:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074773.043799:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074773.043847:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074773.043872:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c400/0x7fe2323adfd53a85 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc2a expref: 14 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074773.043891:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074773.043944:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616844672:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 181us (618us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.043951:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7029 00000100:00100000:0.0:1769074773.043968:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7030 00000100:00100000:0.0:1769074773.043975:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616844544:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074773.043983:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074773.043992:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 23 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074773.044002:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 23 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074773.044015:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676030400/0x7fe2323adfd53a8c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc31 expref: 23 pid: 28955 timeout: 1860 lvb_type: 0 00010000:00010000:0.0:1769074773.044025:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.044027:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.044035:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.044044:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.044086:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074773.044101:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625141376:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074773.044147:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074773.056585:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.056919:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625141376:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074773.056945:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074773.057016:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616844544:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13038us (13612us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.057032:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7030 00000100:00100000:2.0:1769074773.057182:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844800 00000100:00100000:2.0:1769074773.057194:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844800 00002000:00100000:2.0:1769074773.057261:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (471040->716799) req@ffff89b5461e5180 x1855007616844800/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.057278:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5180 with x1855007616844800 ext(471040->716799) 00010000:00010000:2.0:1769074773.057293:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 13 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074773.057338:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 13 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074773.057357:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5461e5180 00000100:00100000:2.0:1769074773.057407:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 770 00000100:00100000:2.0:1769074773.057421:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616844800:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074773.064063:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.064222:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616844928 00000100:00100000:1.0:1769074773.064234:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616844928 00002000:00100000:1.0:1769074773.064296:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3280896->3284991) req@ffff89b543859f80 x1855007616844928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.064317:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859f80 with x1855007616844928 ext(3280896->3284991) 00002000:00010000:1.0:1769074773.064324:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543859f80 00000100:00100000:1.0:1769074773.064359:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8556 00000100:00100000:1.0:1769074773.064370:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616844928:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074773.064732:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616844928:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 362us (681us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074773.064740:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8556 00002000:00100000:1.0:1769074773.064748:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3280896->3284991) req@ffff89b543859f80 x1855007616844928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/4536 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.064756:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859f80 with x1855007616844928 ext(3280896->3284991) 00002000:00010000:1.0:1769074773.064760:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543859f80 00000100:00100000:2.0:1769074773.066292:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616844800:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 8861us (9722us total) trans 17179874511 rc 0/0 00000100:00100000:2.0:1769074773.066316:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 770 00002000:00100000:2.0:1769074773.066329:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bc:0x0] (471040->716799) req@ffff89b5461e5180 x1855007616844800/t17179874511(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.066338:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e5180 with x1855007616844800 ext(471040->716799) 00010000:00010000:2.0:1769074773.066347:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:2.0:1769074773.066367:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00002000:00010000:2.0:1769074773.066375:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5461e5180 00000100:00100000:2.0:1769074773.075652:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.075772:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616845056 00000100:00100000:1.0:1769074773.075783:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616845056 00010000:00010000:1.0:1769074773.075862:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00000100:00100000:1.0:1769074773.075905:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7031 00000100:00100000:1.0:1769074773.075914:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616845056:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074773.075929:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074773.075949:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074773.075963:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074773.075988:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910ca00/0x7fe2323adfd53a54 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 471040->716799) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbf9 expref: 15 pid: 9048 timeout: 1860 lvb_type: 0 00010000:00010000:1.0:1769074773.076005:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.076007:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.076014:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.076025:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.076065:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.076080:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625141504:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074773.076185:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.083986:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625141504:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074773.084013:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074773.084088:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616845056:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8172us (8444us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.084102:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7031 00000100:00100000:3.0:1769074773.096776:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.099450:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.099777:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.100125:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.100166:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616845440 00000100:00100000:3.0:1769074773.100179:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616845440 00002000:00100000:3.0:1769074773.100257:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b543858380 x1855007616845440/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.100271:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858380 with x1855007616845440 ext(0->1048575) 00002000:00010000:3.0:1769074773.100277:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543858380 00000100:00100000:3.0:1769074773.100318:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616845568 00000100:00100000:3.0:1769074773.100322:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616845568 00002000:00100000:3.0:1769074773.100341:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54385aa00 x1855007616845568/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 504/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.100354:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385aa00 with x1855007616845568 ext(1048576->2097151) 00002000:00010000:3.0:1769074773.100358:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385aa00 00000100:00100000:3.0:1769074773.100373:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616845696 00000100:00100000:3.0:1769074773.100376:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616845696 00002000:00100000:3.0:1769074773.100388:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54385a680 x1855007616845696/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.100396:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a680 with x1855007616845696 ext(1048576->2097151) 00002000:00010000:3.0:1769074773.100399:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385a680 00000100:00100000:3.0:1769074773.100417:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8557 00000100:00100000:3.0:1769074773.100427:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616845440:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074773.100591:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8558 00000100:00100000:2.0:1769074773.100602:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616845568:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074773.101985:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8559 00000100:00100000:1.0:1769074773.102000:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616845696:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074773.113161:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616845312 00000100:00100000:0.0:1769074773.113197:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616845312 00000100:00100000:0.0:1769074773.113312:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13320 00000100:00100000:0.0:1769074773.113324:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62869:x1855007616845312:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074773.113355:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.113381:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53b34 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bca1 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.113626:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074773.113642:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074773.113664:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074773.113678:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074773.113685:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074773.113691:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.114968:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b49 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.114991:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b49 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115001:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b49 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074773.115015:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074773.115146:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115153:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115159:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115166:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115171:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074773.115176:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115182:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115206:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074773.115209:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074773.115214:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115221:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53b3b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115236:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115241:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115246:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115250:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074773.115253:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115258:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115265:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47400) 00010000:00010000:1.0:1769074773.115268:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074773.115271:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115280:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b42 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074773.115348:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53b34 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bca1 expref: 254 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.115365:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b49 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bca1 expref: 253 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.115373:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074773.115376:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074773.115379:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:1.0:1769074773.115483:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62869:x1855007616845312:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 2160us (18718us total) trans 8589943947 rc 0/0 00000100:00100000:1.0:1769074773.115494:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13320 00000100:00100000:0.0:1769074773.118540:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616845568:12345-192.168.204.60@tcp:3:cat.0 Request processed in 17930us (18752us total) trans 0 rc 647168/647168 00000100:00100000:0.0:1769074773.118554:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8558 00002000:00100000:0.0:1769074773.118565:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54385aa00 x1855007616845568/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 504/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 647168/647168 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.118575:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385aa00 with x1855007616845568 ext(1048576->2097151) 00002000:00010000:0.0:1769074773.118580:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385aa00 00000100:00100000:3.0:1769074773.134664:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616845440:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34216us (35215us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074773.134679:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8557 00002000:00100000:3.0:1769074773.134692:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b543858380 x1855007616845440/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.134702:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858380 with x1855007616845440 ext(0->1048575) 00002000:00010000:3.0:1769074773.134708:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543858380 00000100:00100000:2.0:1769074773.135958:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616845696:12345-192.168.204.60@tcp:3:cat.0 Request processed in 33953us (35823us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074773.135970:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8559 00002000:00100000:2.0:1769074773.135980:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54385a680 x1855007616845696/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.135991:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a680 with x1855007616845696 ext(1048576->2097151) 00002000:00010000:2.0:1769074773.135996:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385a680 00000100:00100000:1.0:1769074773.138781:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.138970:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.139450:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846080 00000100:00100000:0.0:1769074773.139552:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846080 00000100:00100000:0.0:1769074773.139634:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1010 00000100:00100000:0.0:1769074773.139644:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:2409:x1855007616846080:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074773.139673:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074773.139702:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53b49 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bca1 expref: 253 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.139725:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074773.139732:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074773.139862:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+252:2409:x1855007616846080:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 219us (891us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.139871:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1010 00000100:00100000:3.0:1769074773.143551:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616845952 00000100:00100000:3.0:1769074773.143790:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616845952 00000100:00100000:2.0:1769074773.145436:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4668 00000100:00100000:2.0:1769074773.145450:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+252:62869:x1855007616845952:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074773.145992:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62869:x1855007616845952:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 542us (7220us total) trans 8589943948 rc 0/0 00000100:00100000:2.0:1769074773.146002:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4668 00000100:00100000:1.0:1769074773.166029:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.166073:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.166315:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846208 00000100:00100000:0.0:1769074773.166326:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846208 00000100:00100000:0.0:1769074773.166382:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846336 00000100:00100000:0.0:1769074773.166385:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846336 00000100:00100000:0.0:1769074773.166419:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14079 00000100:00100000:0.0:1769074773.166428:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616846208:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.166447:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.166477:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b57 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bca8 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074773.166539:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625141632:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074773.166616:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.166841:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14080 00000100:00100000:3.0:1769074773.166852:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616846336:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074773.166861:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074773.166872:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53b5e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcaf expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074773.166906:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625141760:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074773.166938:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c510c0 going to sleep for 11 seconds 00002000:00010000:0.0:1769074773.169996:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074771 -> 1769074772 00000100:00100000:0.0:1769074773.170027:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625141632:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074773.170069:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b57 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bca8 expref: 23 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.170092:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b57 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bca8 expref: 23 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.170104:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877e00, rc 0) 00000100:00100000:0.0:1769074773.170174:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616846208:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3746us (4153us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074773.170184:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14079 00000100:00100000:0.0:1769074773.174836:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625141760:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074773.174885:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53b5e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcaf expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.174911:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53b5e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcaf expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.174924:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ce00, rc 0) 00000100:00100000:0.0:1769074773.174993:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616846336:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8143us (8918us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074773.175003:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14080 00000100:00100000:1.0:1769074773.191123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.191439:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846464 00000100:00100000:0.0:1769074773.191450:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846464 00000100:00100000:0.0:1769074773.191552:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14081 00000100:00100000:0.0:1769074773.191562:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62867:x1855007616846464:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.191582:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.191605:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcb6 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.191636:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.191653:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.191656:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.191660:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 23 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.191698:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.191712:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 13 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.191725:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625141888:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.191866:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074773.195842:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625141888:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074773.195879:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 23 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.195898:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877e00, rc 0) 00000100:00100000:0.0:1769074773.195949:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62867:x1855007616846464:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4389us (4837us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.195959:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14081 00000100:00100000:2.0:1769074773.201267:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.201536:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846592 00000100:00100000:1.0:1769074773.201547:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846592 00002000:00100000:1.0:1769074773.201601:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3284991) req@ffff89b67ff0fb80 x1855007616846592/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.201612:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0fb80 with x1855007616846592 ext(3145728->3284991) 00010000:00010000:1.0:1769074773.201621:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 14 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.201642:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 14 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:1.0:1769074773.201650:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67ff0fb80 00000100:00100000:1.0:1769074773.201686:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 771 00000100:00100000:1.0:1769074773.201696:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616846592:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074773.206430:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.207465:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846720 00000100:00100000:0.0:1769074773.207478:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846720 00000100:00100000:0.0:1769074773.207541:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14082 00000100:00100000:0.0:1769074773.207552:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616846720:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.207572:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.207591:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcbd expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.207615:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.207618:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.207628:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 17 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.207646:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 17 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.207660:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877400, rc 0) 00000100:00100000:0.0:1769074773.207725:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616846720:12345-192.168.204.60@tcp:101:cat.0 Request processed in 174us (1311us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.207733:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14082 00000100:00100000:2.0:1769074773.208168:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.208251:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.208348:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846848 00000100:00100000:1.0:1769074773.208354:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846848 00000100:00100000:0.0:1769074773.208395:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616846976 00000100:00100000:0.0:1769074773.208400:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616846976 00000100:00080000:2.0:1769074773.208433:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074773.208435:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14083 00000100:00100000:0.0:1769074773.208451:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616846976:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.208461:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.208473:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd53b73 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bccb expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074773.208529:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625142016:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074773.208593:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14084 00000100:00100000:0.0:1769074773.208601:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.208601:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616846848:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074773.209037:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074773.209055:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b7a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcc4 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.209091:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625142144:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074773.209418:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.212337:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616846592:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 10628us (11065us total) trans 17179874512 rc 0/0 00000100:00100000:1.0:1769074773.212350:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 771 00002000:00100000:1.0:1769074773.212365:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3284991) req@ffff89b67ff0fb80 x1855007616846592/t17179874512(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.212375:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0fb80 with x1855007616846592 ext(3145728->3284991) 00010000:00010000:1.0:1769074773.212384:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.212399:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:1.0:1769074773.212406:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67ff0fb80 00000100:00100000:0.0:1769074773.212658:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625142016:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074773.212693:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd53b73 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bccb expref: 18 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.212716:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877000/0x7fe2323adfd53b73 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bccb expref: 18 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.212729:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877000, rc 0) 00000100:00100000:0.0:1769074773.212755:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616846976:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4314us (4504us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074773.212762:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14083 00000100:00100000:1.0:1769074773.214601:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625142144:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074773.214659:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b7a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcc4 expref: 16 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074773.214680:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53b7a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcc4 expref: 16 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074773.214691:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651e00, rc 0) 00000100:00100000:1.0:1769074773.214749:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2408:x1855007616846848:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6149us (6582us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074773.214758:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14084 00000100:00100000:2.0:1769074773.216155:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.216357:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847104 00000100:00100000:3.0:1769074773.216367:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847104 00010000:00010000:3.0:1769074773.216405:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 17 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:3.0:1769074773.216532:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7032 00000100:00100000:3.0:1769074773.216542:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616847104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074773.216556:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074773.216577:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 17 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.216591:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 17 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.216611:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641650800/0x7fe2323adfd53b26 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3284991) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc93 expref: 17 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.216627:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074773.216629:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074773.216636:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.216645:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.216681:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074773.216695:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625142272:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074773.216778:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c510c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074773.223399:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625142272:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074773.223435:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074773.223504:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2409:x1855007616847104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6962us (7354us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.223516:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7032 00000100:00100000:2.0:1769074773.232477:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.232521:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.232743:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847232 00000100:00100000:3.0:1769074773.233281:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847232 00002000:00100000:2.0:1769074773.233565:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b67ff0df80 x1855007616847232/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.233577:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0df80 with x1855007616847232 ext(2097152->3145727) 00002000:00010000:2.0:1769074773.233584:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0df80 00000100:00100000:2.0:1769074773.233671:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847360 00000100:00100000:2.0:1769074773.233677:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847360 00002000:00100000:2.0:1769074773.233697:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b67ff0ea00 x1855007616847360/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.233705:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ea00 with x1855007616847360 ext(0->1048575) 00002000:00010000:2.0:1769074773.233709:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0ea00 00000100:00100000:2.0:1769074773.233732:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8560 00000100:00100000:2.0:1769074773.233740:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616847232:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.234330:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8561 00000100:00100000:3.0:1769074773.234342:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616847360:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074773.236275:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.236331:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847488 00000100:00100000:0.0:1769074773.236342:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847488 00000100:00100000:0.0:1769074773.236401:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14085 00000100:00100000:0.0:1769074773.236410:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+15:62876:x1855007616847488:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074773.236431:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.236600:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcd2 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.236730:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.236740:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.236743:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.236747:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1826816->1884159] (req 1826816->1884159) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.236906:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 20 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.236922:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 20 pid: 28955 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.236937:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625142400:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.237017:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074773.249346:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.250385:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847616 00000100:00100000:1.0:1769074773.250400:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847616 00002000:00100000:1.0:1769074773.250459:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1515520->2084863) req@ffff89b545109180 x1855007616847616/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.250471:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109180 with x1855007616847616 ext(1515520->2084863) 00010000:00010000:1.0:1769074773.250480:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 21 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.250503:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 21 pid: 28955 timeout: 1861 lvb_type: 0 00002000:00010000:1.0:1769074773.250512:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545109180 00000100:00100000:1.0:1769074773.250540:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 772 00000100:00100000:1.0:1769074773.250549:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616847616:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074773.250603:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616847232:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16862us (18136us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074773.250875:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8560 00002000:00100000:3.0:1769074773.250883:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b67ff0df80 x1855007616847232/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.250892:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0df80 with x1855007616847232 ext(2097152->3145727) 00002000:00010000:3.0:1769074773.250896:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0df80 00000100:00100000:3.0:1769074773.250993:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616847360:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16654us (18470us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074773.251002:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8561 00002000:00100000:3.0:1769074773.251008:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b67ff0ea00 x1855007616847360/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.251016:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ea00 with x1855007616847360 ext(0->1048575) 00002000:00010000:3.0:1769074773.251021:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0ea00 00000100:00100000:0.0:1769074773.252141:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625142400:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074773.252169:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1826816->1884159] (req 1826816->1884159) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.252186:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877000, rc 0) 00000100:00100000:0.0:1769074773.252230:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62876:x1855007616847488:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 15820us (15960us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.252238:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14085 00000100:00100000:3.0:1769074773.266048:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.266321:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847744 00000100:00100000:0.0:1769074773.266332:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847744 00000100:00100000:0.0:1769074773.266408:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14086 00000100:00100000:0.0:1769074773.266419:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62867:x1855007616847744:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.266441:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.266464:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcd9 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.266498:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.266506:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.266508:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.266512:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.266551:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.266563:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 15 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.266575:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625142528:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.266683:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.270177:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.270416:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625142528:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074773.270467:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.270493:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:0.0:1769074773.270585:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62867:x1855007616847744:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4167us (4548us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.270594:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14086 00000100:00100000:3.0:1769074773.271703:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.271876:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848000 00000100:00100000:0.0:1769074773.271886:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848000 00000100:00100000:0.0:1769074773.271944:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14087 00000100:00100000:0.0:1769074773.271954:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+16:62873:x1855007616848000:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:1.0:1769074773.271965:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616847872 00000100:00100000:1.0:1769074773.271971:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616847872 00010000:00010000:0.0:1769074773.271975:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.271999:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bce0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00100000:1.0:1769074773.272009:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2822144->3145727) req@ffff89b545389500 x1855007616847872/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.272018:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545389500 with x1855007616847872 ext(2822144->3145727) 00010000:00010000:1.0:1769074773.272026:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.272027:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.272033:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1826816->1884159] (req 1826816->1884159) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.272037:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.272039:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:0.0:1769074773.272043:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074773.272044:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545389500 00000100:00100000:1.0:1769074773.272065:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 773 00000100:00100000:1.0:1769074773.272071:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616847872:12345-192.168.204.60@tcp:4:multiop.0 00010000:00010000:0.0:1769074773.272080:0:36516:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1826816->1884159] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.272093:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.272107:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:0.0:1769074773.272152:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62873:x1855007616848000:12345-192.168.204.60@tcp:101:cat.0 Request processed in 199us (453us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.272158:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14087 00000100:00100000:2.0:1769074773.275029:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616847616:12345-192.168.204.60@tcp:4:cat.0 Request processed in 24469us (25684us total) trans 21474841198 rc 0/0 00000100:00100000:2.0:1769074773.275044:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 772 00002000:00100000:2.0:1769074773.275060:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1515520->2084863) req@ffff89b545109180 x1855007616847616/t21474841198(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.275071:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109180 with x1855007616847616 ext(1515520->2084863) 00010000:00010000:2.0:1769074773.275081:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 24 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.275096:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 24 pid: 28955 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074773.275103:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545109180 00000100:00100000:1.0:1769074773.278747:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616847872:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 6662us (8566us total) trans 21474841199 rc 0/0 00000100:00100000:1.0:1769074773.278759:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 773 00002000:00100000:1.0:1769074773.278774:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2822144->3145727) req@ffff89b545389500 x1855007616847872/t21474841199(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074773.278784:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545389500 with x1855007616847872 ext(2822144->3145727) 00010000:00010000:1.0:1769074773.278822:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.278837:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:1.0:1769074773.278843:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b545389500 00000100:00100000:0.0:1769074773.280588:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.280706:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848128 00000100:00100000:2.0:1769074773.280717:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848128 00010000:00010000:2.0:1769074773.280757:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 24 pid: 28955 timeout: 1861 lvb_type: 0 00000100:00100000:2.0:1769074773.280871:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7033 00000100:00100000:2.0:1769074773.280882:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616848128:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074773.280897:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074773.280919:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 24 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.280932:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 24 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.280960:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53ad2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1515520->1916927) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc5b expref: 24 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.280975:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.280977:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.280985:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.280995:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.281034:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.281045:0:6500:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:2.0:1769074773.281062:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625142784:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074773.281149:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.282123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.282381:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848256 00000100:00100000:0.0:1769074773.282390:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848256 00010000:00010000:0.0:1769074773.282425:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.282462:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7034 00000100:00100000:0.0:1769074773.282470:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6429be050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616848256:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074773.282481:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074773.282502:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.282514:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.282536:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea1400/0x7fe2323adfd53aee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2822144->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc77 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.282548:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.282550:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.282558:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.282568:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.282597:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074773.282610:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625142912:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074773.282715:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.286351:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.286521:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848384 00000100:00100000:3.0:1769074773.286531:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848384 00002000:00100000:3.0:1769074773.286583:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1826816->1830911) req@ffff89b67b412d80 x1855007616848384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.286594:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b412d80 with x1855007616848384 ext(1826816->1830911) 00010000:00010000:3.0:1769074773.286603:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.286618:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:3.0:1769074773.286626:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67b412d80 00000100:00100000:3.0:1769074773.286664:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 774 00000100:00100000:3.0:1769074773.286673:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616848384:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074773.286906:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616848384:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 233us (565us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074773.286913:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 774 00002000:00100000:3.0:1769074773.286920:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1826816->1830911) req@ffff89b67b412d80 x1855007616848384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/4536 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.286928:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b412d80 with x1855007616848384 ext(1826816->1830911) 00010000:00010000:3.0:1769074773.286933:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.286944:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:3.0:1769074773.286951:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67b412d80 00000100:00100000:2.0:1769074773.288284:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625142784:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074773.288324:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074773.288430:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616848128:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7549us (7850us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.288440:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7033 00000100:00100000:0.0:1769074773.288572:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625142912:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074773.288583:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074773.288605:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6429be050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616848256:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6136us (6491us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.288612:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7034 00000100:00100000:3.0:1769074773.292257:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.292368:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848512 00000100:00100000:2.0:1769074773.292378:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848512 00002000:00100000:2.0:1769074773.292429:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1880064->1884159) req@ffff89b54538b100 x1855007616848512/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.292440:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54538b100 with x1855007616848512 ext(1880064->1884159) 00010000:00010000:2.0:1769074773.292449:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.292464:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074773.292472:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54538b100 00000100:00100000:2.0:1769074773.292503:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 775 00000100:00100000:2.0:1769074773.292512:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616848512:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074773.292730:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2406:x1855007616848512:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 218us (485us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074773.292743:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 775 00002000:00100000:2.0:1769074773.292751:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1880064->1884159) req@ffff89b54538b100 x1855007616848512/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/4536 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.292759:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54538b100 with x1855007616848512 ext(1880064->1884159) 00010000:00010000:2.0:1769074773.292765:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.292787:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074773.292803:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54538b100 00000100:00100000:0.0:1769074773.299386:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.299428:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.299526:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848640 00000100:00100000:2.0:1769074773.299538:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848640 00002000:00100000:2.0:1769074773.299591:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5461e7480 x1855007616848640/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.299602:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7480 with x1855007616848640 ext(2097152->4194303) 00002000:00010000:2.0:1769074773.299608:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5461e7480 00000100:00100000:2.0:1769074773.299634:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848768 00000100:00100000:2.0:1769074773.299638:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848768 00002000:00100000:2.0:1769074773.299664:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5461e4700 x1855007616848768/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.299672:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e4700 with x1855007616848768 ext(3145728->4194303) 00002000:00010000:2.0:1769074773.299675:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e4700 00000100:00100000:2.0:1769074773.299704:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8562 00000100:00100000:2.0:1769074773.299714:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616848640:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074773.301106:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8563 00000100:00100000:1.0:1769074773.301118:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616848768:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074773.304285:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.309199:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616848896 00000100:00100000:2.0:1769074773.309225:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616848896 00002000:00100000:2.0:1769074773.309323:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1826816->1884159) req@ffff89b67b410000 x1855007616848896/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.309338:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b410000 with x1855007616848896 ext(1826816->1884159) 00010000:00010000:2.0:1769074773.309354:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.309376:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 18 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074773.309384:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67b410000 00000100:00100000:2.0:1769074773.309450:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 776 00000100:00100000:2.0:1769074773.309463:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616848896:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074773.324705:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616848896:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 15232us (20421us total) trans 21474841200 rc 0/0 00000100:00100000:2.0:1769074773.324722:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 776 00002000:00100000:2.0:1769074773.324735:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2ba:0x0] (1826816->1884159) req@ffff89b67b410000 x1855007616848896/t21474841200(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.324745:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b410000 with x1855007616848896 ext(1826816->1884159) 00010000:00010000:2.0:1769074773.324757:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.324775:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074773.324785:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67b410000 00000100:00100000:2.0:1769074773.335638:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616848640:12345-192.168.204.60@tcp:3:cat.0 Request processed in 35911us (36251us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074773.335662:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8562 00002000:00100000:2.0:1769074773.335675:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5461e7480 x1855007616848640/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.335686:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e7480 with x1855007616848640 ext(2097152->4194303) 00002000:00010000:2.0:1769074773.335692:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5461e7480 00000100:00100000:0.0:1769074773.338069:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.338277:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616849024 00000100:00100000:1.0:1769074773.338291:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616849024 00010000:00010000:1.0:1769074773.338345:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:1.0:1769074773.338433:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7035 00000100:00100000:1.0:1769074773.338461:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616849024:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074773.338478:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074773.338504:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.338523:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.338548:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd53b81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1884159) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd2 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.338566:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.338568:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.338576:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.338586:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.338633:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.338647:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625143040:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074773.338728:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.342736:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616848768:12345-192.168.204.60@tcp:3:cat.0 Request processed in 41608us (43294us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074773.342752:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8563 00002000:00100000:3.0:1769074773.342767:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5461e4700 x1855007616848768/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.342779:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e4700 with x1855007616848768 ext(3145728->4194303) 00002000:00010000:3.0:1769074773.342787:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e4700 00000100:00100000:0.0:1769074773.354420:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.354581:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616849408 00000100:00100000:2.0:1769074773.354594:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616849408 00002000:00100000:2.0:1769074773.354958:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b67ff0ce00 x1855007616849408/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.354970:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ce00 with x1855007616849408 ext(1048576->2097151) 00002000:00010000:2.0:1769074773.354976:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0ce00 00000100:00100000:3.0:1769074773.354987:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.355003:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8564 00000100:00100000:2.0:1769074773.355012:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616849408:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074773.355038:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616849280 00000100:00100000:3.0:1769074773.355043:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616849280 00000100:00100000:3.0:1769074773.355079:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13321 00000100:00100000:3.0:1769074773.355086:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62879:x1855007616849280:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074773.355105:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074773.355133:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53b96 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bce7 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.355462:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625143040:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074773.355488:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074773.355718:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616849024:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17264us (17658us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.355728:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7035 00010000:00010000:3.0:1769074773.355769:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.355785:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356371:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356395:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356404:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356411:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356535:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53bab lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356546:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53bab lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074773.356552:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53bab lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074773.356563:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074773.359536:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359556:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359562:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359570:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359577:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910d800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074773.359584:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359592:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359619:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074773.359622:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074773.359627:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359633:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d800/0x7fe2323adfd53b9d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359649:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359655:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359660:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359665:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074773.359667:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359673:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359679:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6439ad500) 00010000:00010000:2.0:1769074773.359682:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074773.359685:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359690:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910c000/0x7fe2323adfd53ba4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.359753:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910ce00/0x7fe2323adfd53b96 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce7 expref: 256 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.359768:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53bab lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce7 expref: 255 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.359787:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074773.359790:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074773.359792:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910de00, rc 0) 00000100:00100000:2.0:1769074773.359898:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:62879:x1855007616849280:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 4808us (4907us total) trans 8589943949 rc 0/0 00000100:00100000:2.0:1769074773.359907:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13321 00000001:00080000:1.0:1769074773.368363:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943949 is committed 00000100:00100000:1.0:1769074773.370422:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.371109:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616849536 00000100:00100000:3.0:1769074773.371121:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616849536 00000100:00100000:3.0:1769074773.371184:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14088 00000100:00100000:3.0:1769074773.371196:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616849536:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074773.371218:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074773.371243:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcee expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.371275:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074773.371279:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074773.371292:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 24 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074773.371321:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 24 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074773.371334:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910ca00, rc 0) 00000100:00100000:3.0:1769074773.371402:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616849536:12345-192.168.204.60@tcp:101:cat.0 Request processed in 208us (990us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.371410:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14088 00000100:00100000:2.0:1769074773.372437:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616849408:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 17418us (18021us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074773.372450:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8564 00002000:00100000:2.0:1769074773.372965:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b67ff0ce00 x1855007616849408/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.372977:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ce00 with x1855007616849408 ext(1048576->2097151) 00002000:00010000:2.0:1769074773.372983:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0ce00 00000100:00100000:0.0:1769074773.373159:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.373597:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.373893:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616849920 00000100:00100000:3.0:1769074773.373902:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616849920 00000100:00100000:3.0:1769074773.373957:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1011 00000100:00100000:3.0:1769074773.373967:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:2409:x1855007616849920:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074773.373982:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074773.374089:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910de00/0x7fe2323adfd53bab lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce7 expref: 225 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.374111:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074773.374116:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074773.374170:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2409:x1855007616849920:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 205us (573us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.374178:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1011 00000100:00100000:2.0:1769074773.374823:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616849792 00000100:00100000:2.0:1769074773.374830:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616849792 00000100:00100000:2.0:1769074773.374866:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4669 00000100:00100000:2.0:1769074773.374875:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:62879:x1855007616849792:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074773.375124:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:62879:x1855007616849792:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 249us (1965us total) trans 8589943950 rc 0/0 00000100:00100000:2.0:1769074773.375132:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4669 00000100:00100000:0.0:1769074773.391755:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.391940:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850048 00000100:00100000:2.0:1769074773.391954:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850048 00002000:00100000:2.0:1769074773.392016:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5437b5180 x1855007616850048/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.392028:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5180 with x1855007616850048 ext(4194304->4808703) 00002000:00010000:2.0:1769074773.392034:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b5180 00000100:00100000:2.0:1769074773.392124:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8565 00000100:00100000:2.0:1769074773.392135:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616850048:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074773.399768:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.399850:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.400018:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850176 00000100:00100000:0.0:1769074773.400030:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850176 00000100:00100000:0.0:1769074773.400099:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850304 00000100:00100000:0.0:1769074773.400102:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850304 00000100:00100000:0.0:1769074773.400127:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14089 00000100:00100000:0.0:1769074773.400135:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2406:x1855007616850176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.400161:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1769074773.400177:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14090 00000100:00100000:2.0:1769074773.400188:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616850304:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.400196:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcf5 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.400197:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074773.400206:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bcfc expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.400220:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.400220:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.400223:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.400223:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.400232:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 17 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.400232:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 20 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.400248:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 17 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.400248:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 20 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.400262:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9600, rc 0) 00010000:00010000:2.0:1769074773.400262:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866800, rc 0) 00000100:00100000:2.0:1769074773.400415:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616850304:12345-192.168.204.60@tcp:101:cat.0 Request processed in 229us (564us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.400423:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14090 00000100:00100000:0.0:1769074773.400545:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616850176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 406us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.400554:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14089 00010000:00010000:3.0:1769074773.405173:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074773.407325:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616850048:12345-192.168.204.60@tcp:3:cat.0 Request processed in 15180us (15570us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074773.407341:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8565 00002000:00100000:2.0:1769074773.407357:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5437b5180 x1855007616850048/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.407368:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5180 with x1855007616850048 ext(4194304->4808703) 00002000:00010000:2.0:1769074773.407375:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b5180 00000100:00100000:0.0:1769074773.450215:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.450381:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850432 00000100:00100000:2.0:1769074773.450392:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850432 00002000:00100000:2.0:1769074773.450442:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5437b7800 x1855007616850432/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.450454:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b7800 with x1855007616850432 ext(2097152->4194303) 00002000:00010000:2.0:1769074773.450460:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b7800 00000100:00100000:2.0:1769074773.450497:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8566 00000100:00100000:2.0:1769074773.450507:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616850432:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.452893:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.460963:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850560 00000100:00100000:0.0:1769074773.460983:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850560 00002000:00100000:0.0:1769074773.461061:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b545388e00 x1855007616850560/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.461075:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388e00 with x1855007616850560 ext(2097152->4194303) 00002000:00010000:0.0:1769074773.461082:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545388e00 00000100:00100000:0.0:1769074773.461126:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8567 00000100:00100000:0.0:1769074773.461135:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616850560:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.487613:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616850432:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37082us (37386us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074773.487631:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8566 00002000:00100000:3.0:1769074773.487654:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5437b7800 x1855007616850432/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.487664:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b7800 with x1855007616850432 ext(2097152->4194303) 00002000:00010000:3.0:1769074773.487671:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5437b7800 00000100:00100000:3.0:1769074773.490530:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616850560:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29384us (37635us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074773.490546:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8567 00002000:00100000:3.0:1769074773.490560:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b545388e00 x1855007616850560/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.490569:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388e00 with x1855007616850560 ext(2097152->4194303) 00002000:00010000:3.0:1769074773.490575:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545388e00 00000100:00100000:1.0:1769074773.532049:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.532194:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850688 00000100:00100000:0.0:1769074773.532208:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850688 00002000:00100000:0.0:1769074773.532280:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67b411c00 x1855007616850688/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.532304:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b411c00 with x1855007616850688 ext(4194304->4808703) 00002000:00010000:0.0:1769074773.532312:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67b411c00 00000100:00100000:0.0:1769074773.532365:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8568 00000100:00100000:0.0:1769074773.532378:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616850688:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074773.560528:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2406:x1855007616850688:12345-192.168.204.60@tcp:3:cat.0 Request processed in 28141us (28480us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074773.560543:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8568 00002000:00100000:0.0:1769074773.560555:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b67b411c00 x1855007616850688/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.560565:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b411c00 with x1855007616850688 ext(4194304->4808703) 00002000:00010000:0.0:1769074773.560571:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67b411c00 00000100:00100000:1.0:1769074773.724449:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.724582:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616850944 00000100:00100000:2.0:1769074773.724599:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616850944 00000100:00100000:2.0:1769074773.724711:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13322 00000100:00100000:2.0:1769074773.724729:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:62886:x1855007616850944:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074773.724776:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074773.724842:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd03 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.724931:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.724948:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.724961:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.724979:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.724989:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.724996:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725140:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd53be3 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725152:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd53be3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725160:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd53be3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074773.725173:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074773.725283:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725309:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725316:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725323:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725329:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074773.725334:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725342:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725358:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074773.725362:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074773.725369:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725376:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd53bd5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725397:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725403:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725410:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725415:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074773.725419:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725425:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725434:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4000) 00010000:00010000:2.0:1769074773.725437:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074773.725440:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725445:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd53bdc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074773.725493:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd03 expref: 228 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.725507:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd53be3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd03 expref: 227 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.725514:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074773.725516:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074773.725519:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867a00, rc 0) 00000100:00100000:2.0:1769074773.725610:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62886:x1855007616850944:12345-192.168.204.60@tcp:101:cat.0 Request processed in 884us (1168us total) trans 8589943951 rc 0/0 00000100:00100000:2.0:1769074773.725626:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13322 00000100:00100000:1.0:1769074773.744160:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.744429:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.744471:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.745395:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851456 00000100:00100000:2.0:1769074773.745412:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851456 00000100:00100000:2.0:1769074773.745485:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14091 00000100:00100000:2.0:1769074773.745496:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62879:x1855007616851456:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074773.745516:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074773.745534:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd0a expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.745565:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.745574:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.745582:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.745584:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.745589:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1163264->2072575] (req 1163264->2072575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.745635:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.745648:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 1861 lvb_type: 1 00000100:00100000:2.0:1769074773.745660:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625143168:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074773.745773:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.745783:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 19 pid: 28955 timeout: 1861 lvb_type: 0 00000100:00100000:2.0:1769074773.745837:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625143296:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074773.745871:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abbcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.747046:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851328 00000100:00100000:3.0:1769074773.747064:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851328 00000100:00100000:3.0:1769074773.747144:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1012 00000100:00100000:3.0:1769074773.747153:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2407:x1855007616851328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074773.747169:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074773.747210:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd53be3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd03 expref: 227 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.747236:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074773.747243:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074773.747312:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2407:x1855007616851328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 158us (2884us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.747326:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1012 00000100:00100000:0.0:1769074773.753143:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851200 00000100:00100000:0.0:1769074773.753168:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851200 00000100:00100000:0.0:1769074773.753393:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4670 00000100:00100000:0.0:1769074773.753410:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:62886:x1855007616851200:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:0.0:1769074773.753739:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62886:x1855007616851200:12345-192.168.204.60@tcp:35:cat.0 Request processed in 332us (9585us total) trans 8589943952 rc 0/0 00000100:00100000:0.0:1769074773.753749:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4670 00000100:00100000:2.0:1769074773.766272:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625143168:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074773.766300:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625143296:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074773.766332:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1163264->2072575] (req 1163264->2072575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 25 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.766350:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866c00, rc 0) 00000100:00100000:2.0:1769074773.766430:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62879:x1855007616851456:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20934us (21957us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.766439:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14091 00000100:00100000:3.0:1769074773.777154:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.777218:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.777335:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851712 00000100:00100000:0.0:1769074773.777348:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851712 00000100:00100000:0.0:1769074773.777430:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14092 00000100:00100000:0.0:1769074773.777442:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616851712:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.777475:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.777500:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53bf8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd11 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074773.777519:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851584 00000100:00100000:2.0:1769074773.777523:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851584 00010000:00010000:0.0:1769074773.777530:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.777546:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53bf8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd11 expref: 26 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074773.777547:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 20 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.777570:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53bf8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd11 expref: 26 pid: 9048 timeout: 0 lvb_type: 1 00000100:00100000:2.0:1769074773.777580:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7036 00010000:00010000:0.0:1769074773.777587:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:2.0:1769074773.777587:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616851584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074773.777608:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074773.777633:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 20 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.777652:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 20 pid: 28955 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.777653:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616851712:12345-192.168.204.60@tcp:101:cat.0 Request processed in 215us (435us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074773.777660:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14092 00010000:00010000:2.0:1769074773.777670:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd53a46 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bbeb expref: 20 pid: 28955 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.777680:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074773.777703:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2406:x1855007616851584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 117us (563us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.777709:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7036 00000100:00100000:3.0:1769074773.782467:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.788353:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851840 00000100:00100000:2.0:1769074773.788371:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851840 00010000:00010000:2.0:1769074773.788435:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 1861 lvb_type: 1 00000100:00100000:2.0:1769074773.788557:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7037 00000100:00100000:2.0:1769074773.788568:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616851840:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074773.788589:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074773.791538:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 1861 lvb_type: 1 00010000:00010000:2.0:1769074773.791576:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 1861 lvb_type: 1 00010000:00010000:2.0:1769074773.791627:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b676030800/0x7fe2323adfd53a69 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc0e expref: 25 pid: 8396 timeout: 1861 lvb_type: 1 00010000:00010000:2.0:1769074773.791927:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.791932:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074773.791943:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.791956:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.792033:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 24 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074773.792055:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625143424:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074773.792516:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abbcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.803142:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.803319:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616851968 00000100:00100000:0.0:1769074773.803330:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616851968 00002000:00100000:0.0:1769074773.803380:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b545389c00 x1855007616851968/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.803391:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545389c00 with x1855007616851968 ext(1048576->2097151) 00002000:00010000:0.0:1769074773.803397:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545389c00 00000100:00100000:0.0:1769074773.803433:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8569 00000100:00100000:0.0:1769074773.803442:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616851968:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074773.806790:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.807960:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852096 00000100:00100000:3.0:1769074773.807974:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852096 00002000:00100000:3.0:1769074773.808025:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1163264->1167359) req@ffff89b54441ed80 x1855007616852096/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.808036:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ed80 with x1855007616852096 ext(1163264->1167359) 00002000:00010000:3.0:1769074773.808042:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441ed80 00000100:00100000:3.0:1769074773.808081:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8570 00000100:00100000:3.0:1769074773.808092:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616852096:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.808504:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616852096:12345-192.168.204.60@tcp:3:cat.0 Request processed in 404us (1720us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074773.808512:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8570 00002000:00100000:3.0:1769074773.808520:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1163264->1167359) req@ffff89b54441ed80 x1855007616852096/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/4536 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.808528:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441ed80 with x1855007616852096 ext(1163264->1167359) 00002000:00010000:3.0:1769074773.808531:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54441ed80 00000100:00100000:2.0:1769074773.813537:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625143424:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074773.813576:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074773.816169:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616851840:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 27585us (33698us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.816561:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7037 00000100:00100000:2.0:1769074773.821197:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616851968:12345-192.168.204.60@tcp:3:cat.0 Request processed in 17718us (18029us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074773.821219:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8569 00002000:00100000:2.0:1769074773.821244:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b545389c00 x1855007616851968/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.821257:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545389c00 with x1855007616851968 ext(1048576->2097151) 00002000:00010000:2.0:1769074773.821266:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545389c00 00000100:00100000:1.0:1769074773.847578:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.847752:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852224 00000100:00100000:2.0:1769074773.847762:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852224 00002000:00100000:2.0:1769074773.847924:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1163263) req@ffff89b67b412680 x1855007616852224/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.847940:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b412680 with x1855007616852224 ext(1048576->1163263) 00002000:00010000:2.0:1769074773.847947:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b412680 00000100:00100000:3.0:1769074773.848085:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8571 00000100:00100000:3.0:1769074773.848098:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616852224:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.850488:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.856358:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852352 00000100:00100000:0.0:1769074773.856377:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852352 00002000:00100000:0.0:1769074773.856447:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (2068480->2072575) req@ffff89b545388380 x1855007616852352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.856461:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388380 with x1855007616852352 ext(2068480->2072575) 00002000:00010000:0.0:1769074773.856468:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545388380 00000100:00100000:3.0:1769074773.857663:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.857958:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852480 00000100:00100000:1.0:1769074773.857966:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852480 00000100:00100000:1.0:1769074773.858011:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14093 00000100:00100000:1.0:1769074773.858020:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62882:x1855007616852480:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074773.858038:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074773.858057:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd18 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.858090:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.858100:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.858108:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.858111:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.858115:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1757184->2097151] (req 1757184->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.858149:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.858171:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 22 pid: 28954 timeout: 1861 lvb_type: 0 00000100:00100000:1.0:1769074773.858184:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625143552:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074773.858273:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.858283:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 18 pid: 28954 timeout: 1861 lvb_type: 0 00000100:00100000:1.0:1769074773.858306:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625143680:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074773.858330:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.858874:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616852224:12345-192.168.204.60@tcp:3:cat.0 Request processed in 10776us (11303us total) trans 0 rc 114688/114688 00000100:00100000:3.0:1769074773.858885:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8571 00002000:00100000:3.0:1769074773.858894:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1163263) req@ffff89b67b412680 x1855007616852224/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 114688/114688 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.858905:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b412680 with x1855007616852224 ext(1048576->1163263) 00002000:00010000:3.0:1769074773.858910:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b412680 00000100:00100000:3.0:1769074773.858931:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8572 00000100:00100000:3.0:1769074773.858939:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616852352:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.859047:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616852352:12345-192.168.204.60@tcp:3:cat.0 Request processed in 110us (8571us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074773.859054:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8572 00002000:00100000:3.0:1769074773.859060:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (2068480->2072575) req@ffff89b545388380 x1855007616852352/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/4536 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.859069:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545388380 with x1855007616852352 ext(2068480->2072575) 00002000:00010000:3.0:1769074773.859073:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545388380 00000100:00100000:1.0:1769074773.867210:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.867261:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.867305:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852736 00000100:00100000:1.0:1769074773.867317:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852736 00010000:00010000:1.0:1769074773.867370:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 19 pid: 28954 timeout: 1861 lvb_type: 0 00000100:00100000:1.0:1769074773.867409:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7038 00000100:00100000:1.0:1769074773.867419:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616852736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074773.867437:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1769074773.867609:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625143552:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.867623:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625143680:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074773.867665:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1757184->2097151] (req 1757184->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.867682:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70000, rc 0) 00000100:00100000:0.0:1769074773.867751:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62882:x1855007616852480:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 9731us (10096us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.867759:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14093 00000100:00100000:0.0:1769074773.867793:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852608 00000100:00100000:0.0:1769074773.867798:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852608 00000100:00100000:0.0:1769074773.867853:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14094 00000100:00100000:0.0:1769074773.867859:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616852608:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.867873:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.867890:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53c06 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd1f expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074773.867947:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625143808:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074773.868012:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00010000:00010000:1.0:1769074773.868336:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 18 pid: 28954 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.868355:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 18 pid: 28954 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.868381:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd53904 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb04 expref: 18 pid: 28954 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.868394:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074773.868431:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2409:x1855007616852736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1014us (1169us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.868440:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7038 00000100:00100000:0.0:1769074773.874282:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:1.0:1769074773.874397:0:36516:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074771 -> 1769074773 00002000:00010000:1.0:1769074773.874402:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074771 -> 1769074773 00002000:00010000:1.0:1769074773.874404:0:36516:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074771 -> 1769074773 00000100:00100000:1.0:1769074773.874422:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625143808:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074773.874466:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53c06 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd1f expref: 20 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074773.874490:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd53c06 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd1f expref: 20 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074773.874503:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c9c00, rc 0) 00000100:00100000:1.0:1769074773.874562:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616852608:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6701us (7367us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074773.874573:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14094 00000100:00100000:3.0:1769074773.875112:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852992 00000100:00100000:3.0:1769074773.875121:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852992 00002000:00100000:3.0:1769074773.875161:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (2072576->2097151) req@ffff89b54b425180 x1855007616852992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.875172:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425180 with x1855007616852992 ext(2072576->2097151) 00002000:00010000:3.0:1769074773.875177:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425180 00000100:00100000:3.0:1769074773.875209:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8573 00000100:00100000:3.0:1769074773.875219:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616852992:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074773.877234:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074773.877436:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616852864 00000100:00100000:0.0:1769074773.877540:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616852864 00010000:00010000:0.0:1769074773.877584:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 23 pid: 28954 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.877670:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7039 00000100:00100000:0.0:1769074773.877679:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b643b0c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616852864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074773.877694:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074773.877716:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 23 pid: 28954 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.877732:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 23 pid: 28954 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.877755:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53935 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bb2e expref: 23 pid: 28954 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.877783:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.877785:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.877794:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.877827:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.877882:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074773.877898:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625143936:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074773.877998:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.895206:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616852992:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19975us (20920us total) trans 0 rc 24576/24576 00000100:00100000:3.0:1769074773.895222:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8573 00002000:00100000:3.0:1769074773.895236:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (2072576->2097151) req@ffff89b54b425180 x1855007616852992/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 24576/24576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.895248:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425180 with x1855007616852992 ext(2072576->2097151) 00002000:00010000:3.0:1769074773.895254:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425180 00000100:00100000:0.0:1769074773.900868:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625143936:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074773.900903:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074773.900972:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b643b0c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616852864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 23293us (23868us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.900984:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7039 00000100:00100000:1.0:1769074773.907892:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.908402:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853120 00000100:00100000:3.0:1769074773.908416:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853120 00002000:00100000:3.0:1769074773.908494:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1757184->1761279) req@ffff89b67a273b80 x1855007616853120/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.908506:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a273b80 with x1855007616853120 ext(1757184->1761279) 00002000:00010000:3.0:1769074773.908523:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a273b80 00000100:00100000:3.0:1769074773.908563:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8574 00000100:00100000:3.0:1769074773.908574:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616853120:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074773.909473:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+17:2408:x1855007616853120:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 890us (1590us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074773.909484:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8574 00002000:00100000:3.0:1769074773.909494:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1757184->1761279) req@ffff89b67a273b80 x1855007616853120/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/4536 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.909503:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a273b80 with x1855007616853120 ext(1757184->1761279) 00002000:00010000:3.0:1769074773.909520:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a273b80 00000100:00100000:2.0:1769074773.914270:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.914379:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.914485:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853248 00000100:00100000:1.0:1769074773.914498:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853248 00000100:00100000:1.0:1769074773.914571:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853376 00000100:00100000:1.0:1769074773.914574:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853376 00000100:00100000:1.0:1769074773.914606:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14095 00000100:00100000:1.0:1769074773.914617:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62886:x1855007616853248:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074773.914643:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074773.914663:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd26 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.914694:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.914703:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074773.914705:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074773.914710:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.914745:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.914758:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 16 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:1.0:1769074773.914784:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625144064:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074773.914890:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074773.915319:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14096 00000100:00100000:0.0:1769074773.915331:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62889:x1855007616853376:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074773.915342:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.915353:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd2d expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.915373:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 23 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.915381:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.915383:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074773.915386:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.915409:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 23 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.915421:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 23 pid: 9048 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.915431:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625144192:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.915485:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074773.917161:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074773.917194:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625144320:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074773.917255:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074773.917617:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625144320 00000100:00100000:0.0:1769074773.917629:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625144320 00000100:00100000:0.0:1769074773.917688:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2647 00000100:00100000:0.0:1769074773.917698:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625144320:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074773.917737:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6389760 pending 0 free 1445400576 avail 1299161088 00000100:00100000:0.0:1769074773.917823:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625144320:12345-0@lo:41:osp-pre-0-1.0 Request processed in 111us (552us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.917831:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2647 00000100:00100000:0.0:1769074773.917885:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625144320:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074773.918595:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074773.918611:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625144448:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074773.918669:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.919028:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625144832:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.919061:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.919083:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625145344:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.919107:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.919125:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625145856:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.919144:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.919162:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1769074773.919168:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625144704:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919317:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625144576:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919349:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074773.919375:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625145088:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919395:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074773.919411:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625145600:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919430:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074773.919458:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1769074773.919463:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625144960:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919483:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074773.919499:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625145472:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919517:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074773.919543:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625144576 00000100:00100000:2.0:1769074773.919550:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625144576 00000100:00100000:2.0:1769074773.919591:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1769074773.919598:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b08450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625144576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.919645:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b08450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625144576:12345-0@lo:400:kworker.0 Request processed in 46us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.919653:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1769074773.919693:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145344 00000100:00100000:2.0:1769074773.919698:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145344 00000100:00100000:2.0:1769074773.919734:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145856 00000100:00100000:2.0:1769074773.919738:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145856 00000100:00100000:2.0:1769074773.919750:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145088 00000100:00100000:2.0:1769074773.919752:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145088 00000100:00100000:1.0:1769074773.919923:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.919936:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625145216:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.919954:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.919964:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625145728:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.919982:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074773.920008:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145472 00000100:00100000:1.0:1769074773.920011:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145472 00000100:00100000:1.0:1769074773.920031:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625144704 00000100:00100000:1.0:1769074773.920034:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625144704 00000100:00100000:1.0:1769074773.920044:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145216 00000100:00100000:1.0:1769074773.920047:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145216 00000100:00100000:1.0:1769074773.920061:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2312 00000100:00100000:1.0:1769074773.920067:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625145344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.920107:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625145344:12345-0@lo:400:kworker.0 Request processed in 40us (999us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.920114:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2312 00000100:00100000:1.0:1769074773.920128:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2313 00000100:00100000:1.0:1769074773.920133:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625145856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.920158:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625145856:12345-0@lo:400:kworker.0 Request processed in 24us (1012us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.920163:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2313 00000100:00100000:2.0:1769074773.920329:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2314 00000100:00100000:2.0:1769074773.920337:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625145472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920368:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625145472:12345-0@lo:400:kworker.0 Request processed in 33us (850us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.920374:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2314 00000100:00100000:2.0:1769074773.920389:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2315 00000100:00100000:2.0:1769074773.920395:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625144704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920421:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625144704:12345-0@lo:400:kworker.0 Request processed in 27us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.920428:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2315 00000100:00100000:2.0:1769074773.920439:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2316 00000100:00100000:2.0:1769074773.920444:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625145216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920469:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625145216:12345-0@lo:400:kworker.0 Request processed in 25us (514us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.920475:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2316 00000100:00100000:2.0:1769074773.920486:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2317 00000100:00100000:2.0:1769074773.920492:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625145088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920519:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625145088:12345-0@lo:400:kworker.0 Request processed in 27us (1122us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.920528:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2317 00000100:00100000:2.0:1769074773.920546:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625144832 00000100:00100000:2.0:1769074773.920550:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625144832 00000100:00100000:2.0:1769074773.920571:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145600 00000100:00100000:2.0:1769074773.920575:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145600 00000100:00100000:2.0:1769074773.920590:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625144960 00000100:00100000:2.0:1769074773.920593:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625144960 00000100:00100000:2.0:1769074773.920609:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625145728 00000100:00100000:2.0:1769074773.920612:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625145728 00000100:00100000:2.0:1769074773.920629:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1444 00000100:00100000:2.0:1769074773.920634:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625144832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920665:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625144832:12345-0@lo:400:kworker.0 Request processed in 30us (1603us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.920671:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1444 00000100:00100000:2.0:1769074773.920682:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1445 00000100:00100000:2.0:1769074773.920688:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625145600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920714:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625145600:12345-0@lo:400:kworker.0 Request processed in 26us (1283us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.920719:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1445 00000100:00100000:2.0:1769074773.920729:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1446 00000100:00100000:2.0:1769074773.920735:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625144960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.920758:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625144960:12345-0@lo:400:kworker.0 Request processed in 24us (1274us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.921015:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1447 00000100:00100000:3.0:1769074773.921024:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625145728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074773.921057:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625145728:12345-0@lo:400:kworker.0 Request processed in 35us (1074us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.921063:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1447 00000100:00100000:3.0:1769074773.921088:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625144448 00000100:00100000:3.0:1769074773.921093:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625144448 00000100:00100000:3.0:1769074773.921118:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2648 00000100:00100000:3.0:1769074773.921124:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625144448:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074773.921149:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6438912 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074773.921194:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625144448:12345-0@lo:41:osp-pre-1-0.0 Request processed in 70us (2524us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.921200:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2648 00000100:00100000:3.0:1769074773.921267:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625144960:0@lo:400:kworker.0 00000100:00100000:3.0:1769074773.921280:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625145472:0@lo:400:kworker.0 00000100:00100000:3.0:1769074773.921301:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625144576:0@lo:400:kworker.0 00000100:00100000:3.0:1769074773.921312:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625145088:0@lo:400:kworker.0 00000100:00100000:3.0:1769074773.921321:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625145600:0@lo:400:kworker.0 00000100:00100000:3.0:1769074773.921337:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625144448:0@lo:41:osp-pre-1-0.0 00000100:00080000:3.0:1769074773.921381:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074773.921385:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074773.921388:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074773.921390:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074773.921626:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625144704:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.921650:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625145216:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.921661:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625145728:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.921670:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625144832:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.921680:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625145344:0@lo:400:kworker.0 00000100:00100000:1.0:1769074773.921689:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625145856:0@lo:400:kworker.0 00000100:00100000:2.0:1769074773.922125:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1446 00000100:00100000:2.0:1769074773.924059:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.925583:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625144064:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074773.925624:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074773.925657:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70400, rc 0) 00000100:00100000:2.0:1769074773.925719:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62886:x1855007616853248:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11104us (11464us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074773.925728:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14095 00000100:00100000:2.0:1769074773.925758:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853504 00000100:00100000:2.0:1769074773.925774:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853504 00000100:00100000:0.0:1769074773.925861:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14097 00000100:00100000:0.0:1769074773.925872:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+17:62882:x1855007616853504:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074773.925887:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074773.925904:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd34 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.925930:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.925939:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.925947:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.925949:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074773.925953:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->2801663] (req 2097152->2801663) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.925980:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 18 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074773.925991:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 18 pid: 36516 timeout: 1861 lvb_type: 1 00000100:00100000:0.0:1769074773.926004:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625145984:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074773.926064:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.926074:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 22 pid: 9048 timeout: 1861 lvb_type: 0 00000100:00100000:0.0:1769074773.926082:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625146112:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.926106:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4f00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.926892:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625144192:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074773.926917:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.926931:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8e00, rc 0) 00000100:00100000:3.0:1769074773.926972:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62889:x1855007616853376:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11644us (12592us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.926978:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14096 00000100:00100000:1.0:1769074773.928404:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.928955:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853632 00000100:00100000:3.0:1769074773.928964:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853632 00002000:00100000:3.0:1769074773.929019:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1757184->2097151) req@ffff89b54475a680 x1855007616853632/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.929032:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475a680 with x1855007616853632 ext(1757184->2097151) 00010000:00010000:3.0:1769074773.929042:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.929058:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:3.0:1769074773.929067:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54475a680 00000100:00100000:3.0:1769074773.929384:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 777 00000100:00100000:0.0:1769074773.929469:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616853632:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074773.944506:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.946377:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853760 00000100:00100000:3.0:1769074773.946392:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853760 00002000:00100000:3.0:1769074773.946643:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1163264->2072575) req@ffff89b549201880 x1855007616853760/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.946655:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549201880 with x1855007616853760 ext(1163264->2072575) 00010000:00010000:3.0:1769074773.946666:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 24 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.946681:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 24 pid: 9048 timeout: 1861 lvb_type: 0 00002000:00010000:3.0:1769074773.946688:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549201880 00000100:00100000:3.0:1769074773.946722:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 778 00000100:00100000:3.0:1769074773.946733:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007616853760:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074773.948924:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625145984:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074773.948938:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625146112:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074773.948974:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->2801663] (req 2097152->2801663) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074773.948994:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8000, rc 0) 00000100:00100000:0.0:1769074773.949049:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62882:x1855007616853504:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 23178us (25003us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074773.949058:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14097 00000100:00100000:0.0:1769074773.951214:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616853632:12345-192.168.204.60@tcp:4:cat.0 Request processed in 21808us (22804us total) trans 17179874513 rc 0/0 00000100:00100000:0.0:1769074773.951233:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 777 00002000:00100000:0.0:1769074773.951249:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1757184->2097151) req@ffff89b54475a680 x1855007616853632/t17179874513(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074773.951260:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54475a680 with x1855007616853632 ext(1757184->2097151) 00010000:00010000:0.0:1769074773.951270:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:0.0:1769074773.951284:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 20 pid: 36516 timeout: 1861 lvb_type: 0 00002000:00010000:0.0:1769074773.951302:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54475a680 00000100:00100000:2.0:1769074773.967280:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074773.967450:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074773.967637:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616853888 00000100:00100000:1.0:1769074773.967650:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616853888 00010000:00010000:1.0:1769074773.967711:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 20 pid: 36516 timeout: 1861 lvb_type: 1 00000100:00100000:1.0:1769074773.967761:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854016 00000100:00100000:1.0:1769074773.967765:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854016 00010000:00010000:1.0:1769074773.967779:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 23 pid: 9048 timeout: 1861 lvb_type: 0 00000100:00100000:1.0:1769074773.967801:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7040 00000100:00100000:1.0:1769074773.967830:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616853888:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074773.967850:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074773.967878:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 20 pid: 36516 timeout: 1861 lvb_type: 1 00010000:00010000:1.0:1769074773.967895:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 20 pid: 36516 timeout: 1861 lvb_type: 1 00010000:00010000:1.0:1769074773.967923:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9600/0x7fe2323adfd53bc0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcf5 expref: 20 pid: 36516 timeout: 1861 lvb_type: 1 00010000:00010000:1.0:1769074773.967946:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074773.968000:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2408:x1855007616853888:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 173us (732us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.968006:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7040 00000100:00100000:1.0:1769074773.968022:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7041 00000100:00100000:1.0:1769074773.968026:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616854016:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074773.968032:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074773.968037:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 23 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.968044:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 23 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.968053:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877e00/0x7fe2323adfd53b65 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcb6 expref: 23 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:1.0:1769074773.968328:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.968334:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074773.968344:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.968354:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074773.968395:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 18 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074773.968409:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625146240:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074773.968477:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074773.969566:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.969973:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854144 00000100:00100000:3.0:1769074773.969987:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854144 00010000:00010000:3.0:1769074773.970036:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00000100:00100000:3.0:1769074773.970086:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7042 00000100:00100000:3.0:1769074773.970098:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2407:x1855007616854144:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074773.970113:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1769074773.970130:0:36300:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074770 -> 1769074773 00002000:00010000:3.0:1769074773.970133:0:36300:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074770 -> 1769074773 00010000:00010000:3.0:1769074773.970138:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.970149:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.970171:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70000/0x7fe2323adfd53bff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1757184->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd18 expref: 19 pid: 36516 timeout: 1861 lvb_type: 0 00010000:00010000:3.0:1769074773.970190:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074773.970192:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074773.970199:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.970209:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074773.970245:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074773.970258:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625146368:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074773.971313:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074773.980541:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625146368:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074773.980569:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074773.980648:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616854144:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10550us (11086us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074773.980659:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7042 00000100:00100000:2.0:1769074773.988384:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616853760:12345-192.168.204.60@tcp:4:cat.0 Request processed in 41636us (43871us total) trans 21474841201 rc 0/0 00000100:00100000:2.0:1769074773.988399:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 778 00002000:00100000:2.0:1769074773.988415:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1163264->2072575) req@ffff89b549201880 x1855007616853760/t21474841201(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/448 e 0 to 0 dl 1769074784 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074773.988426:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549201880 with x1855007616853760 ext(1163264->2072575) 00010000:00010000:2.0:1769074773.988447:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 26 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074773.988464:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 26 pid: 9048 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074773.988482:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b549201880 00000100:00100000:1.0:1769074773.988643:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625146240:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074773.988677:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074773.988706:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616854016:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 20678us (21257us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074773.988715:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7041 00000100:00100000:2.0:1769074773.996447:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074773.996648:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854272 00000100:00100000:3.0:1769074773.996659:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854272 00002000:00100000:3.0:1769074773.996724:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cfef800 x1855007616854272/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/0 e 0 to 0 dl 1769074784 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074773.996735:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfef800 with x1855007616854272 ext(1048576->2097151) 00002000:00010000:3.0:1769074773.996741:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfef800 00000100:00100000:3.0:1769074773.996788:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8575 00000100:00100000:3.0:1769074773.996799:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616854272:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074774.008695:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.008993:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854400 00000100:00100000:1.0:1769074774.009006:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854400 00002000:00100000:1.0:1769074774.009060:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2797568->2801663) req@ffff89b54cfedc00 x1855007616854400/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.009082:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfedc00 with x1855007616854400 ext(2797568->2801663) 00002000:00010000:1.0:1769074774.009089:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfedc00 00000100:00100000:1.0:1769074774.009128:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8576 00000100:00100000:1.0:1769074774.009138:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616854400:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074774.009333:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2407:x1855007616854400:12345-192.168.204.60@tcp:3:cat.0 Request processed in 194us (648us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074774.009340:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8576 00002000:00100000:1.0:1769074774.009346:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2797568->2801663) req@ffff89b54cfedc00 x1855007616854400/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.009353:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfedc00 with x1855007616854400 ext(2797568->2801663) 00002000:00010000:1.0:1769074774.009356:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfedc00 00000100:00100000:0.0:1769074774.012199:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.012379:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854528 00000100:00100000:2.0:1769074774.012392:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854528 00010000:00010000:2.0:1769074774.012461:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 26 pid: 9048 timeout: 1861 lvb_type: 0 00000100:00100000:2.0:1769074774.012503:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7043 00000100:00100000:2.0:1769074774.012514:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616854528:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074774.012529:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074774.012549:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 26 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.012563:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 26 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.012587:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd53bf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1163264->2072575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd0a expref: 26 pid: 9048 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.012602:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.012605:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.012612:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.012622:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.012658:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 19 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074774.012677:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625146496:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074774.012755:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00080000:3.0:1769074774.013018:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074774.021775:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625146496:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074774.021837:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074774.021892:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616854528:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9379us (9707us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.021904:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7043 00000100:00100000:0.0:1769074774.023735:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.025406:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854656 00000100:00100000:3.0:1769074774.025423:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854656 00002000:00100000:3.0:1769074774.025486:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67b411f80 x1855007616854656/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.025497:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b411f80 with x1855007616854656 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.025503:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b411f80 00000100:00100000:3.0:1769074774.025607:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8577 00000100:00100000:3.0:1769074774.025618:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616854656:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.028530:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616854272:12345-192.168.204.60@tcp:3:cat.0 Request processed in 31721us (32097us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074774.028544:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8575 00002000:00100000:3.0:1769074774.028557:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cfef800 x1855007616854272/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:309/0 lens 488/440 e 0 to 0 dl 1769074784 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.028567:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfef800 with x1855007616854272 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.028574:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfef800 00000100:00100000:0.0:1769074774.057684:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.057899:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854784 00000100:00100000:2.0:1769074774.057911:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854784 00000100:00100000:2.0:1769074774.057976:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14098 00000100:00100000:2.0:1769074774.057989:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62886:x1855007616854784:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074774.058009:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.058032:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd3b expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.058070:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.058079:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074774.058081:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074774.058086:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.058332:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 17 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.058549:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 17 pid: 8396 timeout: 1861 lvb_type: 0 00000100:00100000:2.0:1769074774.058569:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625146624:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.058682:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.065570:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625146624:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.065610:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.065644:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b6200, rc 0) 00000100:00100000:2.0:1769074774.065715:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62886:x1855007616854784:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7729us (8050us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.065724:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14098 00000100:00100000:3.0:1769074774.067359:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616854656:12345-192.168.204.60@tcp:3:cat.0 Request processed in 41731us (43620us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074774.067374:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8577 00002000:00100000:3.0:1769074774.067390:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67b411f80 x1855007616854656/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.067400:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b411f80 with x1855007616854656 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.067408:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b411f80 00000100:00100000:1.0:1769074774.072166:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.072354:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616854912 00000100:00100000:2.0:1769074774.072366:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616854912 00002000:00100000:2.0:1769074774.072425:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2801663) req@ffff89b544759f80 x1855007616854912/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.072436:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759f80 with x1855007616854912 ext(2097152->2801663) 00010000:00010000:2.0:1769074774.072457:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 18 pid: 8396 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.072476:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 18 pid: 8396 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074774.072490:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b544759f80 00000100:00100000:2.0:1769074774.072868:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 779 00000100:00100000:2.0:1769074774.072880:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+18:2408:x1855007616854912:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074774.097779:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.097966:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616855040 00000100:00100000:0.0:1769074774.097980:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616855040 00000100:00100000:0.0:1769074774.098087:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14099 00000100:00100000:0.0:1769074774.098099:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62889:x1855007616855040:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074774.098133:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074774.098165:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd42 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.098204:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.098207:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.098219:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.098239:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.098258:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:0.0:1769074774.098351:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62889:x1855007616855040:12345-192.168.204.60@tcp:101:cat.0 Request processed in 254us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.098359:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14099 00000100:00100000:2.0:1769074774.101365:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616854912:12345-192.168.204.60@tcp:4:cat.0 Request processed in 28474us (29197us total) trans 17179874514 rc 0/0 00000100:00100000:2.0:1769074774.101382:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 779 00002000:00100000:2.0:1769074774.101394:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2801663) req@ffff89b544759f80 x1855007616854912/t17179874514(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/448 e 0 to 0 dl 1769074785 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.101405:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b544759f80 with x1855007616854912 ext(2097152->2801663) 00010000:00010000:2.0:1769074774.101415:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 21 pid: 8396 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.101434:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 21 pid: 8396 timeout: 1861 lvb_type: 0 00002000:00010000:2.0:1769074774.101452:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b544759f80 00000100:00100000:1.0:1769074774.110359:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.110898:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616855168 00000100:00100000:2.0:1769074774.110910:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616855168 00010000:00010000:2.0:1769074774.110953:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 21 pid: 8396 timeout: 1861 lvb_type: 0 00000100:00100000:2.0:1769074774.111005:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7044 00000100:00100000:2.0:1769074774.111015:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616855168:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074774.111039:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1769074774.111076:0:6500:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074772 -> 1769074773 00002000:00010000:2.0:1769074774.111079:0:6500:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074772 -> 1769074773 00010000:00010000:2.0:1769074774.111086:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 21 pid: 8396 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.111111:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 21 pid: 8396 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.111141:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8000/0x7fe2323adfd53c1b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2801663) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd34 expref: 21 pid: 8396 timeout: 1861 lvb_type: 0 00010000:00010000:2.0:1769074774.111159:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.111161:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.111169:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.111179:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.111226:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074774.111241:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625146752:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074774.111343:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.119335:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.119423:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616855296 00000100:00100000:3.0:1769074774.119447:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616855296 00002000:00100000:3.0:1769074774.119502:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cfed500 x1855007616855296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.119516:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed500 with x1855007616855296 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.119522:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfed500 00000100:00100000:3.0:1769074774.119562:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8578 00000100:00100000:3.0:1769074774.119571:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616855296:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074774.120642:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625146752:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074774.120676:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074774.120747:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616855168:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9733us (10400us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.120758:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7044 00000100:00100000:2.0:1769074774.142711:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.142865:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616855424 00000100:00100000:3.0:1769074774.142877:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616855424 00002000:00100000:3.0:1769074774.142929:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54cfeca80 x1855007616855424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.142941:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfeca80 with x1855007616855424 ext(2097152->4194303) 00002000:00010000:3.0:1769074774.142948:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfeca80 00000100:00100000:3.0:1769074774.142975:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8579 00000100:00100000:3.0:1769074774.142986:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616855424:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074774.154000:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616855296:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34411us (34656us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074774.154020:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8578 00002000:00100000:0.0:1769074774.154039:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54cfed500 x1855007616855296/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.154052:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed500 with x1855007616855296 ext(1048576->2097151) 00002000:00010000:0.0:1769074774.154059:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfed500 00000100:00100000:1.0:1769074774.165301:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.165671:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616855680 00000100:00100000:2.0:1769074774.165683:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616855680 00000100:00100000:2.0:1769074774.165763:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13323 00000100:00100000:2.0:1769074774.165775:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:62893:x1855007616855680:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074774.165801:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.165870:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53c30 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd49 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.165952:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.165970:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.165986:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166002:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166008:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166014:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166177:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53c45 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166189:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53c45 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166196:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53c45 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074774.166206:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074774.166307:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166315:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166321:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166327:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166332:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074774.166336:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166344:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166357:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074774.166361:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074774.166365:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166372:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877000/0x7fe2323adfd53c37 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166388:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166393:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166402:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166407:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074774.166409:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166413:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166419:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4000) 00010000:00010000:2.0:1769074774.166421:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074774.166424:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166429:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd53c3e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074774.166467:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877800/0x7fe2323adfd53c30 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd49 expref: 230 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.166480:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53c45 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd49 expref: 229 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.166488:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074774.166490:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074774.166492:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876600, rc 0) 00000100:00100000:2.0:1769074774.166552:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62893:x1855007616855680:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 781us (1272us total) trans 8589943953 rc 0/0 00000100:00100000:2.0:1769074774.166561:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13323 00000100:00080000:3.0:1769074774.170761:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074774.174890:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616855424:12345-192.168.204.60@tcp:3:cat.0 Request processed in 31890us (32174us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074774.174906:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8579 00002000:00100000:2.0:1769074774.174922:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54cfeca80 x1855007616855424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.174932:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfeca80 with x1855007616855424 ext(2097152->4194303) 00002000:00010000:2.0:1769074774.174939:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfeca80 00000100:00100000:0.0:1769074774.175528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.175572:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.175596:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856192 00000100:00100000:0.0:1769074774.175606:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856192 00000100:00100000:1.0:1769074774.175645:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.175669:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616855808 00000100:00100000:1.0:1769074774.175674:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616855808 00000100:00100000:1.0:1769074774.175723:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1013 00000100:00100000:1.0:1769074774.175735:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:2409:x1855007616855808:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074774.175752:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074774.175785:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876600/0x7fe2323adfd53c45 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd49 expref: 229 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.175827:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074774.175833:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074774.175878:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:2409:x1855007616855808:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 145us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.175885:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1013 00000100:00100000:2.0:1769074774.176134:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856064 00000100:00100000:2.0:1769074774.176139:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856064 00000100:00100000:2.0:1769074774.176186:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4671 00000100:00100000:2.0:1769074774.176193:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:62893:x1855007616856064:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074774.176627:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62893:x1855007616856064:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 433us (1101us total) trans 8589943954 rc 0/0 00000100:00100000:2.0:1769074774.176731:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4671 00000100:00100000:0.0:1769074774.176866:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14100 00000100:00100000:0.0:1769074774.176877:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62889:x1855007616856192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074774.176890:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074774.176912:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd50 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.176936:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.176939:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.176949:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.176969:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.176982:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:0.0:1769074774.177006:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62889:x1855007616856192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 132us (1434us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.177012:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14100 00000100:00100000:1.0:1769074774.186591:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.187365:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856320 00000100:00100000:0.0:1769074774.187380:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856320 00000100:00100000:0.0:1769074774.187601:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14101 00000100:00100000:0.0:1769074774.187612:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62893:x1855007616856320:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074774.187635:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074774.187657:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd57 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.187692:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.187702:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.187710:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.187712:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.187717:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1372160->1495039] (req 1372160->1495039) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 24 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.187752:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.187767:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 20 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.187780:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625146880:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074774.188068:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.188078:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 24 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.188088:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147008:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074774.188119:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.195525:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.195608:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.195640:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856448 00000100:00100000:0.0:1769074774.195652:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856448 00000100:00100000:3.0:1769074774.195686:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856576 00000100:00100000:3.0:1769074774.195691:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856576 00002000:00100000:0.0:1769074774.195715:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54cfed180 x1855007616856448/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.195726:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed180 with x1855007616856448 ext(2097152->4194303) 00002000:00010000:0.0:1769074774.195731:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfed180 00000100:00100000:3.0:1769074774.195736:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14102 00000100:00100000:3.0:1769074774.195747:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616856576:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:0.0:1769074774.195756:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8580 00000100:00100000:0.0:1769074774.195763:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616856448:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:3.0:1769074774.195764:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074774.195787:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd5e expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.195847:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.195850:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.195862:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 26 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074774.195880:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 26 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074774.195892:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c400, rc 0) 00000100:00100000:3.0:1769074774.195939:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616856576:12345-192.168.204.60@tcp:101:cat.0 Request processed in 195us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.195946:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14102 00000100:00100000:1.0:1769074774.196659:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625146880:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074774.204151:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.204226:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.204249:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856704 00000100:00100000:0.0:1769074774.204260:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856704 00010000:00010000:0.0:1769074774.204314:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 23 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.204365:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856832 00000100:00100000:0.0:1769074774.204370:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856832 00010000:00010000:0.0:1769074774.204386:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.204411:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7045 00000100:00100000:0.0:1769074774.204421:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b643b0fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2406:x1855007616856704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074774.204436:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074774.204474:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 23 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.204487:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 23 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.204513:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd53c29 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd42 expref: 23 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.204531:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074774.204588:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b643b0fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616856704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 167us (447us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.204596:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7045 00000100:00100000:0.0:1769074774.204657:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7046 00000100:00100000:0.0:1769074774.204664:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616856832:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074774.204673:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074774.204682:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.204692:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.204706:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.204708:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.204716:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.204727:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.204762:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074774.204776:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625147136:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074774.204852:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.207341:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.207842:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616856960 00000100:00100000:3.0:1769074774.207852:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616856960 00002000:00100000:3.0:1769074774.207902:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54759fb80 x1855007616856960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.207914:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759fb80 with x1855007616856960 ext(4194304->4808703) 00002000:00010000:3.0:1769074774.207921:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759fb80 00000100:00100000:3.0:1769074774.208034:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8581 00000100:00100000:3.0:1769074774.208044:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616856960:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:1.0:1769074774.208212:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53c0d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd26 expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.208233:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147008:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074774.208279:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.208310:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:1.0:1769074774.208392:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62893:x1855007616856320:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 20782us (21809us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.208402:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14101 00000100:00100000:3.0:1769074774.210990:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625147136:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074774.211020:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074774.211176:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616856832:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6508us (6945us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.211187:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7046 00000100:00100000:0.0:1769074774.215553:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616856448:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19774us (20024us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074774.215568:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8580 00002000:00100000:0.0:1769074774.215581:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54cfed180 x1855007616856448/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.215596:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed180 with x1855007616856448 ext(2097152->4194303) 00002000:00010000:0.0:1769074774.215603:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cfed180 00000100:00100000:3.0:1769074774.217646:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616856960:12345-192.168.204.60@tcp:3:cat.0 Request processed in 9589us (10305us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074774.217662:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8581 00002000:00100000:3.0:1769074774.217675:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54759fb80 x1855007616856960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.217686:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759fb80 with x1855007616856960 ext(4194304->4808703) 00002000:00010000:3.0:1769074774.217693:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759fb80 00000100:00100000:2.0:1769074774.218186:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.218273:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857088 00000100:00100000:3.0:1769074774.218293:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857088 00002000:00100000:3.0:1769074774.218329:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1372160->1376255) req@ffff89b54759df80 x1855007616857088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.218338:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759df80 with x1855007616857088 ext(1372160->1376255) 00002000:00010000:3.0:1769074774.218342:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759df80 00000100:00100000:3.0:1769074774.218427:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8582 00000100:00100000:3.0:1769074774.218437:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616857088:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.218617:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616857088:12345-192.168.204.60@tcp:3:cat.0 Request processed in 180us (431us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074774.218624:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8582 00002000:00100000:3.0:1769074774.218630:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1372160->1376255) req@ffff89b54759df80 x1855007616857088/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.218638:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759df80 with x1855007616857088 ext(1372160->1376255) 00002000:00010000:3.0:1769074774.218642:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759df80 00000100:00100000:2.0:1769074774.224959:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.227236:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857216 00000100:00100000:3.0:1769074774.227259:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857216 00002000:00100000:3.0:1769074774.227340:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1490944->1495039) req@ffff89b54759ce00 x1855007616857216/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.227353:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759ce00 with x1855007616857216 ext(1490944->1495039) 00002000:00010000:3.0:1769074774.227360:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759ce00 00000100:00100000:3.0:1769074774.227397:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8583 00000100:00100000:3.0:1769074774.227408:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616857216:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.227607:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616857216:12345-192.168.204.60@tcp:3:cat.0 Request processed in 201us (2666us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074774.227615:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8583 00002000:00100000:3.0:1769074774.227622:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1490944->1495039) req@ffff89b54759ce00 x1855007616857216/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.227630:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54759ce00 with x1855007616857216 ext(1490944->1495039) 00002000:00010000:3.0:1769074774.227633:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54759ce00 00000100:00100000:2.0:1769074774.234855:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.234995:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857344 00000100:00100000:1.0:1769074774.235006:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857344 00000100:00100000:1.0:1769074774.235069:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14103 00000100:00100000:1.0:1769074774.235078:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616857344:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.235096:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.235115:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c68 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd65 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.235176:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147264:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074774.235267:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00002000:00010000:1.0:1769074774.239500:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074770 -> 1769074773 00000100:00100000:1.0:1769074774.239534:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147264:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074774.239570:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c68 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd65 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.239592:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c68 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd65 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.239606:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:1.0:1769074774.239694:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616857344:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4615us (4893us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074774.239704:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14103 00000100:00100000:2.0:1769074774.248653:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.249214:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857472 00000100:00100000:1.0:1769074774.249229:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857472 00002000:00100000:1.0:1769074774.249295:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54c844000 x1855007616857472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.249307:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c844000 with x1855007616857472 ext(4194304->4808703) 00002000:00010000:1.0:1769074774.249313:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c844000 00000100:00100000:1.0:1769074774.249349:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8584 00000100:00100000:1.0:1769074774.249358:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616857472:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074774.259415:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616857472:12345-192.168.204.60@tcp:3:cat.0 Request processed in 10041us (10758us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074774.259432:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8584 00002000:00100000:1.0:1769074774.259448:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b54c844000 x1855007616857472/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.259458:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c844000 with x1855007616857472 ext(4194304->4808703) 00002000:00010000:1.0:1769074774.259466:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54c844000 00000100:00100000:2.0:1769074774.266217:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.266309:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857600 00000100:00100000:1.0:1769074774.266324:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857600 00000100:00100000:1.0:1769074774.266388:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14104 00000100:00100000:1.0:1769074774.266400:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616857600:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.266421:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.266438:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c6f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd6c expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.266493:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147392:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074774.266594:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074774.271366:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147392:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074774.271419:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c6f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd6c expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.271451:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c6f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd6c expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.271467:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:1.0:1769074774.271564:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616857600:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5165us (5353us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074774.271575:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14104 00000100:00100000:3.0:1769074774.278668:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.278841:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857728 00000100:00100000:1.0:1769074774.278853:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857728 00000100:00100000:1.0:1769074774.278916:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14105 00000100:00100000:1.0:1769074774.278925:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616857728:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.278943:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.278961:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd73 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.279023:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147520:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074774.279465:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074774.286480:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147520:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074774.286520:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c76 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd73 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.286544:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c76 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd73 expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.286562:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:1.0:1769074774.287395:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2406:x1855007616857728:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8462us (8734us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074774.287408:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14105 00000100:00100000:2.0:1769074774.327985:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.328072:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857856 00000100:00100000:1.0:1769074774.328085:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857856 00000100:00100000:1.0:1769074774.328142:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14106 00000100:00100000:1.0:1769074774.328150:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62897:x1855007616857856:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074774.328171:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.328191:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd7a expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.328223:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.328228:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.328230:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.328233:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [2023424->2097151] (req 2023424->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.328262:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.328271:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 22 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.328300:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147648:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074774.328372:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.331195:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.331303:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616857984 00000100:00100000:0.0:1769074774.331313:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616857984 00002000:00100000:0.0:1769074774.331363:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1372160->1495039) req@ffff89b67fb95500 x1855007616857984/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.331374:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95500 with x1855007616857984 ext(1372160->1495039) 00010000:00010000:0.0:1769074774.331383:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.331398:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:0.0:1769074774.331405:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67fb95500 00000100:00100000:0.0:1769074774.331441:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 780 00000100:00100000:0.0:1769074774.331450:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616857984:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074774.334691:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147648:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074774.334726:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [2023424->2097151] (req 2023424->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.334746:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:2.0:1769074774.335001:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62897:x1855007616857856:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6850us (7023us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.335015:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14106 00000100:00100000:0.0:1769074774.337139:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616857984:12345-192.168.204.60@tcp:4:cat.0 Request processed in 5681us (5944us total) trans 17179874515 rc 0/0 00000100:00100000:0.0:1769074774.337154:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 780 00002000:00100000:0.0:1769074774.337167:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1372160->1495039) req@ffff89b67fb95500 x1855007616857984/t17179874515(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/448 e 0 to 0 dl 1769074785 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.337177:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb95500 with x1855007616857984 ext(1372160->1495039) 00010000:00010000:0.0:1769074774.337186:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.337205:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:0.0:1769074774.337212:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67fb95500 00000100:00100000:3.0:1769074774.339511:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.339611:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616858112 00000100:00100000:0.0:1769074774.339622:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616858112 00010000:00010000:0.0:1769074774.339732:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.339766:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7047 00000100:00100000:0.0:1769074774.339773:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616858112:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074774.339787:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074774.339878:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.339889:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.339910:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd53c5a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1372160->1495039) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd57 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.339923:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.339925:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.339931:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.339942:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.339974:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074774.339983:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625147776:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074774.340102:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1769074774.345639:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.345790:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625147776:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074774.345838:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074774.345894:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616858112:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6120us (6393us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.345905:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7047 00000100:00100000:1.0:1769074774.346053:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616858240 00000100:00100000:1.0:1769074774.346061:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616858240 00002000:00100000:1.0:1769074774.346100:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (2023424->2027519) req@ffff89b54d1df100 x1855007616858240/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.346111:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1df100 with x1855007616858240 ext(2023424->2027519) 00002000:00010000:1.0:1769074774.346116:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1df100 00000100:00100000:1.0:1769074774.346147:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8585 00000100:00100000:1.0:1769074774.346155:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616858240:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074774.346303:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616858240:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 149us (676us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074774.346309:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8585 00002000:00100000:1.0:1769074774.346315:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (2023424->2027519) req@ffff89b54d1df100 x1855007616858240/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.346323:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1df100 with x1855007616858240 ext(2023424->2027519) 00002000:00010000:1.0:1769074774.346326:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1df100 00000100:00100000:3.0:1769074774.354522:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.354651:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616858368 00000100:00100000:2.0:1769074774.354661:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616858368 00000100:00100000:2.0:1769074774.354726:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14107 00000100:00100000:2.0:1769074774.354735:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62897:x1855007616858368:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074774.354755:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.354773:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd81 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.354855:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.354862:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.354869:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.354870:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.354874:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->2453503] (req 2097152->2453503) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.354910:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.354920:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.354932:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147904:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.355006:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.355013:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 23 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.355018:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625148032:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.355032:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074774.360754:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.361115:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616858496 00000100:00100000:0.0:1769074774.361126:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616858496 00010000:00010000:0.0:1769074774.361171:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 24 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.361214:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7048 00000100:00100000:0.0:1769074774.361224:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616858496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074774.361240:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074774.361260:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 24 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.361276:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 24 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.361394:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074774.361473:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616858496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 250us (730us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.361480:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7048 00000100:00100000:2.0:1769074774.361972:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625147904:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.361988:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b6200/0x7fe2323adfd53c22 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd3b expref: 23 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.362001:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625148032:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.362032:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->2453503] (req 2097152->2453503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.362044:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d96e00, rc 0) 00000100:00100000:2.0:1769074774.362088:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62897:x1855007616858368:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 7354us (7587us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.362094:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14107 00000100:00100000:3.0:1769074774.363332:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.363571:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616858624 00000100:00100000:2.0:1769074774.363585:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616858624 00010000:00010000:2.0:1769074774.363712:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.363850:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7049 00000100:00100000:2.0:1769074774.363864:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2409:x1855007616858624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074774.363882:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074774.363910:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074774.363924:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074774.363951:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd53c53 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd50 expref: 22 pid: 8396 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074774.363968:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.363970:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.363979:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.363990:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.364035:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074774.364050:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625148160:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074774.364142:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.369353:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625148160:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074774.369379:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074774.369634:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616858624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5763us (6299us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.369645:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7049 00000100:00100000:1.0:1769074774.379461:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.379564:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616858752 00000100:00100000:2.0:1769074774.379574:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616858752 00002000:00100000:2.0:1769074774.379640:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2449408->2453503) req@ffff89b5460eed80 x1855007616858752/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.379652:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460eed80 with x1855007616858752 ext(2449408->2453503) 00002000:00010000:2.0:1769074774.379658:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5460eed80 00000100:00100000:2.0:1769074774.379689:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8586 00000100:00100000:2.0:1769074774.379698:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616858752:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074774.380258:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616858752:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 558us (804us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074774.380268:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8586 00002000:00100000:2.0:1769074774.380275:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2449408->2453503) req@ffff89b5460eed80 x1855007616858752/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.380294:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460eed80 with x1855007616858752 ext(2449408->2453503) 00002000:00010000:2.0:1769074774.380299:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5460eed80 00000100:00100000:3.0:1769074774.383610:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.384221:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859008 00000100:00100000:1.0:1769074774.384235:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859008 00000100:00100000:1.0:1769074774.384353:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13324 00000100:00100000:1.0:1769074774.384363:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:62901:x1855007616859008:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.384388:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.384408:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53c8b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd88 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.384493:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384507:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384518:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384531:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384537:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384544:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384662:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53ca0 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384672:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53ca0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384679:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53ca0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074774.384689:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074774.384766:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384773:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384779:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384786:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384792:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5491fb800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074774.384796:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384803:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384841:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074774.384844:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074774.384847:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384853:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53c92 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384867:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384872:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384882:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384887:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5491fa400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074774.384889:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384894:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384900:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074774.384902:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074774.384905:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384910:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53c99 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.384949:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fba00/0x7fe2323adfd53c8b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd88 expref: 232 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.384961:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53ca0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd88 expref: 231 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.384968:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074774.384971:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074774.384973:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:1.0:1769074774.385055:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62901:x1855007616859008:12345-192.168.204.60@tcp:101:cat.0 Request processed in 694us (1460us total) trans 8589943955 rc 0/0 00000100:00100000:1.0:1769074774.385062:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13324 00000100:00100000:3.0:1769074774.392594:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.392645:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.392747:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859392 00000100:00100000:2.0:1769074774.392758:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859392 00000100:00100000:2.0:1769074774.392865:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1014 00000100:00100000:2.0:1769074774.392875:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:2406:x1855007616859392:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074774.392895:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074774.392925:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53ca0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd88 expref: 231 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.392950:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074774.392954:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074774.393003:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:2406:x1855007616859392:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 129us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.393011:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1014 00000100:00100000:1.0:1769074774.393242:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859264 00000100:00100000:1.0:1769074774.393248:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859264 00000100:00100000:1.0:1769074774.393293:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4672 00000100:00100000:1.0:1769074774.393301:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:62901:x1855007616859264:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074774.393540:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62901:x1855007616859264:12345-192.168.204.60@tcp:35:cat.0 Request processed in 239us (893us total) trans 8589943956 rc 0/0 00000100:00100000:1.0:1769074774.393767:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4672 00000100:00100000:3.0:1769074774.404008:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.404136:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859520 00000100:00100000:2.0:1769074774.404147:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859520 00000100:00100000:2.0:1769074774.404211:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14108 00000100:00100000:2.0:1769074774.404220:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616859520:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074774.404239:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.404259:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd53cae lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd8f expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074774.404334:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625148288:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074774.404416:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074774.406456:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.406635:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859648 00000100:00100000:1.0:1769074774.406644:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859648 00000100:00100000:1.0:1769074774.406700:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14109 00000100:00100000:1.0:1769074774.406710:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616859648:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.406727:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.406748:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cb5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd96 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.406848:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625148416:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074774.406965:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00002000:00010000:2.0:1769074774.409770:0:27843:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074773 -> 1769074774 00002000:00010000:2.0:1769074774.409780:0:27843:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074773 -> 1769074774 00000100:00100000:2.0:1769074774.409802:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625148288:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074774.409876:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd53cae lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd8f expref: 26 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074774.409897:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd53cae lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd8f expref: 26 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074774.409910:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97800, rc 0) 00000100:00100000:2.0:1769074774.409975:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616859520:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5755us (5977us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074774.409986:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14108 00002000:00010000:1.0:1769074774.412216:0:36516:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime: 1769074773 -> 1769074774 00002000:00010000:1.0:1769074774.412228:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074772 -> 1769074773 00002000:00010000:1.0:1769074774.412230:0:36516:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074773 -> 1769074774 00000100:00100000:1.0:1769074774.412254:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625148416:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074774.412301:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cb5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd96 expref: 24 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.412322:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cb5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd96 expref: 24 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074774.412338:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:1.0:1769074774.412413:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616859648:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5695us (5965us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074774.412422:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14109 00010000:00010000:0.0:1769074774.429129:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074774.432183:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625148544:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074774.432272:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074774.432338:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1769074774.432346:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625148672:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074774.432368:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074774.432549:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625148544 00000100:00100000:0.0:1769074774.432559:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625148544 00000100:00100000:0.0:1769074774.432632:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625148672 00000100:00100000:0.0:1769074774.432636:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625148672 00000100:00100000:0.0:1769074774.432660:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1164 00000100:00100000:0.0:1769074774.432668:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625148544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074774.432691:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 98304 granted 1096394048 pending 3391488 free 3900067840 avail 3668545536 00000100:00100000:0.0:1769074774.432742:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625148544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (470us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.432749:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1164 00000100:00100000:0.0:1769074774.432765:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1165 00000100:00100000:0.0:1769074774.432770:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625148672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074774.432788:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1091089728 pending 4452352 free 3905998848 avail 3674476544 00000100:00100000:0.0:1769074774.432859:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625148672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (489us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.432864:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1165 00000100:00100000:0.0:1769074774.432919:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625148672:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074774.432934:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625148544:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074774.435415:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074774.439274:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625148800:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074774.439393:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074774.442022:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.442135:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859776 00000100:00100000:3.0:1769074774.442146:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859776 00002000:00100000:3.0:1769074774.442203:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54cfed880 x1855007616859776/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.442214:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed880 with x1855007616859776 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.442221:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfed880 00000100:00100000:3.0:1769074774.442260:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8587 00000100:00100000:3.0:1769074774.442271:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616859776:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074774.447218:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625148800 00000100:00100000:2.0:1769074774.447232:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625148800 00000100:00100000:2.0:1769074774.447299:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1166 00000100:00100000:2.0:1769074774.447309:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625148800:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074774.447332:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1091089728 pending 4452352 free 3905998848 avail 3674476544 00000100:00100000:2.0:1769074774.447404:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625148800:12345-0@lo:13:osp-pre-0-1.0 Request processed in 97us (8012us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.447411:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1166 00000100:00100000:1.0:1769074774.447676:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625148800:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074774.471699:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616859776:12345-192.168.204.60@tcp:3:cat.0 Request processed in 29414us (29676us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074774.471716:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8587 00002000:00100000:2.0:1769074774.471731:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54cfed880 x1855007616859776/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.471752:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cfed880 with x1855007616859776 ext(1048576->2097151) 00002000:00010000:2.0:1769074774.471759:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cfed880 00000100:00100000:0.0:1769074774.486163:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.486273:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616859904 00000100:00100000:1.0:1769074774.486295:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616859904 00000100:00100000:1.0:1769074774.486362:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14110 00000100:00100000:1.0:1769074774.486372:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:62901:x1855007616859904:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.486391:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.486412:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bd9d expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.486455:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.486465:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.486467:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.486472:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.486512:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.486526:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 20 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.486540:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625148928:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074774.486643:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074774.489337:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625148928:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074774.489374:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.489396:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:1.0:1769074774.489470:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62901:x1855007616859904:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3098us (3317us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.489479:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14110 00000100:00100000:3.0:1769074774.494091:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.494441:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616860032 00000100:00100000:2.0:1769074774.494464:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616860032 00002000:00100000:2.0:1769074774.494543:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (2023424->2097151) req@ffff89b67a272d80 x1855007616860032/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.494555:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a272d80 with x1855007616860032 ext(2023424->2097151) 00010000:00010000:2.0:1769074774.494566:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 21 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074774.494588:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 21 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:2.0:1769074774.494596:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a272d80 00000100:00100000:2.0:1769074774.494639:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 781 00000100:00100000:2.0:1769074774.494657:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616860032:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074774.499954:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.500225:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616860288 00000100:00100000:1.0:1769074774.500239:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616860288 00000100:00100000:1.0:1769074774.500331:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13325 00000100:00100000:1.0:1769074774.500341:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:62907:x1855007616860288:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074774.500366:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.500385:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53cc3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bda4 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.500469:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500483:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500494:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500517:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500524:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500530:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500627:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd53cd8 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500640:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd53cd8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500647:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd53cd8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074774.500668:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074774.500727:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500733:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500737:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500742:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500747:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5491fb800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074774.500752:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500758:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500771:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074774.500775:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074774.500782:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500788:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53cca lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500801:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500890:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500895:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500899:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5491fae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074774.500902:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500909:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500916:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074774.500919:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074774.500922:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500932:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fae00/0x7fe2323adfd53cd1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.500979:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd53cc3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bda4 expref: 234 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.500993:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd53cd8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bda4 expref: 233 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.501000:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074774.501002:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074774.501005:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602ae00, rc 0) 00000100:00100000:1.0:1769074774.501075:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62907:x1855007616860288:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 734us (1129us total) trans 8589943957 rc 0/0 00000100:00100000:1.0:1769074774.501084:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13325 00000100:00100000:3.0:1769074774.501869:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616860032:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 7214us (7789us total) trans 17179874516 rc 0/0 00000100:00100000:3.0:1769074774.501881:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 781 00002000:00100000:3.0:1769074774.501892:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (2023424->2097151) req@ffff89b67a272d80 x1855007616860032/t17179874516(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/448 e 0 to 0 dl 1769074785 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.501901:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a272d80 with x1855007616860032 ext(2023424->2097151) 00010000:00010000:3.0:1769074774.501912:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.501934:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:3.0:1769074774.501941:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67a272d80 00000100:00100000:3.0:1769074774.506152:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.506380:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616860544 00000100:00100000:0.0:1769074774.506394:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616860544 00010000:00010000:0.0:1769074774.506454:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.506504:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7050 00000100:00100000:0.0:1769074774.506514:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2407:x1855007616860544:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074774.506528:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074774.506554:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.506568:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.506595:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd53c7d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd7a expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.506613:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.506615:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.506636:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.506647:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.506694:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074774.506710:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625149056:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074774.506849:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074774.511523:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.511586:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616860672 00000100:00100000:1.0:1769074774.511596:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616860672 00000100:00100000:1.0:1769074774.511666:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14111 00000100:00100000:1.0:1769074774.511677:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62908:x1855007616860672:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.511696:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1769074774.511702:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625149056:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074774.511714:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdab expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.511724:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074774.511740:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.511742:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.511749:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.511769:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074774.511780:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2407:x1855007616860544:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5269us (5642us total) trans 0 rc 0/0 00010000:00010000:1.0:1769074774.511782:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602b800, rc 0) 00000100:00100000:3.0:1769074774.511788:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7050 00000100:00100000:1.0:1769074774.511803:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+23:62908:x1855007616860672:12345-192.168.204.60@tcp:101:cat.0 Request processed in 129us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.511829:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14111 00000100:00100000:3.0:1769074774.515945:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.516101:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616860928 00000100:00100000:2.0:1769074774.516110:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616860928 00000100:00100000:2.0:1769074774.516174:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4673 00000100:00100000:2.0:1769074774.516185:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c7c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62907:x1855007616860928:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074774.516497:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c7c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62907:x1855007616860928:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 311us (559us total) trans 8589943958 rc 0/0 00000100:00100000:2.0:1769074774.516511:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4673 00000100:00100000:0.0:1769074774.522570:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.522620:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.522777:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616860416 00000100:00100000:3.0:1769074774.522787:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616860416 00000100:00100000:3.0:1769074774.522878:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1015 00000100:00100000:3.0:1769074774.522887:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:2406:x1855007616860416:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074774.522903:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074774.522927:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd53cd8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bda4 expref: 234 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.522946:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074774.522951:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074774.523015:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:2406:x1855007616860416:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 129us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.523023:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1015 00000100:00100000:2.0:1769074774.523236:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861056 00000100:00100000:2.0:1769074774.523242:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861056 00002000:00100000:2.0:1769074774.523287:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5497aaa00 x1855007616861056/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.523298:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aaa00 with x1855007616861056 ext(1048576->2097151) 00002000:00010000:2.0:1769074774.523302:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aaa00 00000100:00100000:2.0:1769074774.523326:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8588 00000100:00100000:2.0:1769074774.523334:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2407:x1855007616861056:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074774.524637:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.524712:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861184 00000100:00100000:3.0:1769074774.524720:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861184 00002000:00100000:3.0:1769074774.524752:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5446c0000 x1855007616861184/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.524762:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0000 with x1855007616861184 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.524766:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c0000 00000100:00100000:3.0:1769074774.524793:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8589 00000100:00100000:3.0:1769074774.524801:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616861184:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.536962:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.538741:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616861184:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13930us (14109us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074774.538754:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8589 00002000:00100000:3.0:1769074774.538766:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5446c0000 x1855007616861184/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.538776:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5446c0000 with x1855007616861184 ext(1048576->2097151) 00002000:00010000:3.0:1769074774.538782:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5446c0000 00000100:00100000:0.0:1769074774.539684:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5497aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2407:x1855007616861056:12345-192.168.204.60@tcp:3:cat.0 Request processed in 16347us (17059us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074774.539692:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8588 00002000:00100000:0.0:1769074774.539698:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5497aaa00 x1855007616861056/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.539704:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5497aaa00 with x1855007616861056 ext(1048576->2097151) 00002000:00010000:0.0:1769074774.539708:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5497aaa00 00000100:00100000:2.0:1769074774.542424:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861312 00000100:00100000:2.0:1769074774.542448:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861312 00000100:00100000:2.0:1769074774.542525:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14112 00000100:00100000:2.0:1769074774.542535:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62907:x1855007616861312:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074774.542553:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.542570:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdb2 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.542603:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.542613:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 19 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074774.542621:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.542632:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.542637:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3260416->3379199] (req 3260416->3379199) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.542671:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 19 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074774.542683:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 19 pid: 27843 timeout: 1862 lvb_type: 1 00000100:00100000:2.0:1769074774.542696:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625149184:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.542803:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.542836:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.542845:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625149312:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.542873:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074774.547703:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.547792:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861440 00000100:00100000:1.0:1769074774.547820:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861440 00000100:00100000:1.0:1769074774.547877:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14113 00000100:00100000:1.0:1769074774.547887:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:62901:x1855007616861440:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.547904:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.547922:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdb9 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.547953:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.547961:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.547963:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.547968:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.548001:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 22 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.548019:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 22 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.548032:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625149440:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074774.548119:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074774.549155:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625149184:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074774.549170:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625149312:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074774.549199:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3260416->3379199] (req 3260416->3379199) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.549217:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97800, rc 0) 00000100:00100000:3.0:1769074774.549269:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62907:x1855007616861312:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6735us (12318us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.549287:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14112 00000100:00100000:1.0:1769074774.554295:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625149440:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074774.554347:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.554370:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:1.0:1769074774.554447:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62901:x1855007616861440:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6560us (6753us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.554457:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14113 00000100:00100000:1.0:1769074774.556681:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.556792:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861696 00000100:00100000:0.0:1769074774.556856:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861696 00002000:00100000:0.0:1769074774.556923:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2453503) req@ffff89b5460ee680 x1855007616861696/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.556934:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ee680 with x1855007616861696 ext(2097152->2453503) 00010000:00010000:0.0:1769074774.556945:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.556963:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 23 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:0.0:1769074774.556971:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5460ee680 00000100:00100000:0.0:1769074774.557010:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 782 00000100:00100000:0.0:1769074774.557021:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2406:x1855007616861696:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074774.561494:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.561586:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.561964:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861824 00000100:00100000:3.0:1769074774.561975:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861824 00010000:00010000:3.0:1769074774.562013:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:3.0:1769074774.562052:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861568 00000100:00100000:3.0:1769074774.562056:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861568 00010000:00010000:3.0:1769074774.562068:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 20 pid: 27843 timeout: 1862 lvb_type: 1 00000100:00100000:3.0:1769074774.562100:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7051 00000100:00100000:3.0:1769074774.562109:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616861824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074774.562123:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074774.562143:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.562154:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.562175:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876200/0x7fe2323adfd53b88 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcd9 expref: 27 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.562192:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074774.562246:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616861824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 137us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.562254:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7051 00000100:00100000:3.0:1769074774.562271:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7052 00000100:00100000:3.0:1769074774.562288:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616861568:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074774.562296:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074774.562305:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 20 pid: 27843 timeout: 1862 lvb_type: 1 00010000:00010000:3.0:1769074774.562315:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 20 pid: 27843 timeout: 1862 lvb_type: 1 00010000:00010000:3.0:1769074774.562327:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866800/0x7fe2323adfd53bc7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcfc expref: 20 pid: 27843 timeout: 1862 lvb_type: 1 00010000:00010000:3.0:1769074774.562336:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.562339:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.562346:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.562357:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.562398:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074774.562413:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625149568:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074774.562468:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074774.566135:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.566291:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616861952 00000100:00100000:1.0:1769074774.566301:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616861952 00000100:00100000:1.0:1769074774.566393:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14114 00000100:00100000:1.0:1769074774.566403:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+25:62908:x1855007616861952:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074774.566422:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.566450:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdc0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.566483:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074774.566489:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2453504->3145727] (req 2453504->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.566501:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2453504->3145727] (req 2453504->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.566513:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8800, rc 0) 00000100:00100000:1.0:1769074774.566588:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+26:62908:x1855007616861952:12345-192.168.204.60@tcp:101:cat.0 Request processed in 186us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.566595:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14114 00000100:00100000:3.0:1769074774.571054:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625149568:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074774.571296:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074774.571354:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+19:2409:x1855007616861568:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9075us (9766us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.571367:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7052 00000100:00100000:0.0:1769074774.571962:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.572789:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862080 00000100:00100000:3.0:1769074774.572799:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862080 00002000:00100000:3.0:1769074774.572859:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (3260416->3264511) req@ffff89b54ce9ce00 x1855007616862080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.572871:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ce00 with x1855007616862080 ext(3260416->3264511) 00002000:00010000:3.0:1769074774.572876:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9ce00 00000100:00100000:3.0:1769074774.572899:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8590 00000100:00100000:3.0:1769074774.572908:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616862080:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.573035:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616862080:12345-192.168.204.60@tcp:3:cat.0 Request processed in 129us (1076us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074774.573041:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8590 00002000:00100000:3.0:1769074774.573045:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (3260416->3264511) req@ffff89b54ce9ce00 x1855007616862080/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.573051:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ce00 with x1855007616862080 ext(3260416->3264511) 00002000:00010000:3.0:1769074774.573054:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9ce00 00000100:00100000:3.0:1769074774.575960:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2406:x1855007616861696:12345-192.168.204.60@tcp:4:cat.0 Request processed in 18924us (19272us total) trans 17179874517 rc 0/0 00000100:00100000:3.0:1769074774.575975:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 782 00002000:00100000:3.0:1769074774.575991:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2453503) req@ffff89b5460ee680 x1855007616861696/t17179874517(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/448 e 0 to 0 dl 1769074785 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.576002:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ee680 with x1855007616861696 ext(2097152->2453503) 00010000:00010000:3.0:1769074774.576013:0:24028:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.576034:0:24028:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:3.0:1769074774.576043:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5460ee680 00000100:00100000:2.0:1769074774.577362:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.577478:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862208 00000100:00100000:3.0:1769074774.577489:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862208 00002000:00100000:3.0:1769074774.577635:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (3375104->3379199) req@ffff89b67fb97100 x1855007616862208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.577724:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb97100 with x1855007616862208 ext(3375104->3379199) 00002000:00010000:3.0:1769074774.577731:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fb97100 00000100:00100000:3.0:1769074774.577845:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8591 00000100:00100000:3.0:1769074774.577854:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616862208:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.578023:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616862208:12345-192.168.204.60@tcp:3:cat.0 Request processed in 169us (667us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074774.578030:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8591 00002000:00100000:3.0:1769074774.578036:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x2bc:0x0] (3375104->3379199) req@ffff89b67fb97100 x1855007616862208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.578044:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb97100 with x1855007616862208 ext(3375104->3379199) 00002000:00010000:3.0:1769074774.578046:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fb97100 00000100:00100000:2.0:1769074774.585494:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.585732:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862336 00000100:00100000:3.0:1769074774.585743:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862336 00010000:00010000:3.0:1769074774.585786:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:3.0:1769074774.585861:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7053 00000100:00100000:3.0:1769074774.585870:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2406:x1855007616862336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074774.585885:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074774.585906:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.585918:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.585942:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96e00/0x7fe2323adfd53c84 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2453503) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd81 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074774.585959:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.585961:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.585969:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.585979:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.585989:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.585991:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074774.585995:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.586003:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074774.586036:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074774.586049:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625149696:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074774.586210:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074774.586220:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625149824:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074774.586244:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074774.589290:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625149696:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074774.589310:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625149824:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074774.589337:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074774.589409:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2406:x1855007616862336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3537us (3926us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.589422:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7053 00000100:00100000:2.0:1769074774.595719:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.596320:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862464 00000100:00100000:3.0:1769074774.596334:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862464 00002000:00100000:3.0:1769074774.597327:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2453504->3145727) req@ffff89b67fb94380 x1855007616862464/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074774.597341:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb94380 with x1855007616862464 ext(2453504->3145727) 00002000:00010000:3.0:1769074774.597348:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fb94380 00000100:00100000:0.0:1769074774.597471:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8592 00000100:00100000:0.0:1769074774.597488:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2408:x1855007616862464:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074774.607741:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2408:x1855007616862464:12345-192.168.204.60@tcp:3:cat.0 Request processed in 10245us (12016us total) trans 0 rc 692224/692224 00000100:00100000:0.0:1769074774.607759:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8592 00002000:00100000:0.0:1769074774.607775:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2453504->3145727) req@ffff89b67fb94380 x1855007616862464/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 692224/692224 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.607794:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb94380 with x1855007616862464 ext(2453504->3145727) 00002000:00010000:0.0:1769074774.607802:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fb94380 00000100:00100000:3.0:1769074774.624261:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.624307:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.624342:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862592 00000100:00100000:0.0:1769074774.624354:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862592 00000100:00100000:3.0:1769074774.624364:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00100000:0.0:1769074774.624410:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b67a273800 x1855007616862592/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.624422:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a273800 with x1855007616862592 ext(2097152->4194303) 00002000:00010000:0.0:1769074774.624429:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a273800 00000100:00100000:2.0:1769074774.624484:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862848 00000100:00100000:2.0:1769074774.624490:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862848 00000100:00100000:2.0:1769074774.624546:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14115 00000100:00100000:2.0:1769074774.624556:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+19:62908:x1855007616862848:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074774.624571:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.624591:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdc7 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.624638:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.624648:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074774.624651:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074774.624655:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 20 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.624692:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 25 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.624712:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 25 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.624727:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625149952:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.624852:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074774.625035:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862720 00000100:00100000:1.0:1769074774.625041:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862720 00002000:00100000:1.0:1769074774.625064:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b67a273100 x1855007616862720/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 504/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.625073:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a273100 with x1855007616862720 ext(2097152->4194303) 00010000:00010000:1.0:1769074774.625078:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.625089:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 26 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:1.0:1769074774.625095:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a273100 00000100:00100000:1.0:1769074774.625117:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 783 00000100:00100000:1.0:1769074774.625126:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616862720:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074774.626017:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8593 00000100:00100000:0.0:1769074774.626029:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616862592:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074774.646000:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625149952:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.646048:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 20 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.646072:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b679924a00, rc 0) 00000100:00100000:2.0:1769074774.650916:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62908:x1855007616862848:12345-192.168.204.60@tcp:101:cat.0 Request processed in 26342us (26532us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.650936:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14115 00000100:00100000:3.0:1769074774.653898:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.654788:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616862976 00000100:00100000:2.0:1769074774.654802:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616862976 00002000:00100000:2.0:1769074774.654887:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3260416->3379199) req@ffff89b54d1dce00 x1855007616862976/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.654899:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dce00 with x1855007616862976 ext(3260416->3379199) 00010000:00010000:2.0:1769074774.654909:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 28 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074774.654925:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 28 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:2.0:1769074774.654939:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d1dce00 00000100:00100000:2.0:1769074774.654977:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 784 00000100:00100000:2.0:1769074774.654987:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616862976:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074774.655967:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616862720:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30838us (31654us total) trans 0 rc 1978368/1978368 00000100:00100000:1.0:1769074774.655978:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 783 00002000:00100000:1.0:1769074774.655988:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b67a273100 x1855007616862720/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 504/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:H/600/0 rc 1978368/1978368 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.655997:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a273100 with x1855007616862720 ext(2097152->4194303) 00010000:00010000:1.0:1769074774.656005:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 28 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.656019:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 28 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:1.0:1769074774.656026:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67a273100 00000100:00100000:0.0:1769074774.677096:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616862592:12345-192.168.204.60@tcp:3:cat.0 Request processed in 51055us (52829us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074774.677113:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8593 00002000:00100000:0.0:1769074774.677127:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b67a273800 x1855007616862592/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.677137:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a273800 with x1855007616862592 ext(2097152->4194303) 00002000:00010000:0.0:1769074774.677145:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a273800 00000100:00100000:1.0:1769074774.678877:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616862976:12345-192.168.204.60@tcp:4:cat.0 Request processed in 23877us (30437us total) trans 21474841202 rc 0/0 00000100:00100000:1.0:1769074774.678896:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 784 00002000:00100000:1.0:1769074774.678912:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3260416->3379199) req@ffff89b54d1dce00 x1855007616862976/t21474841202(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/448 e 0 to 0 dl 1769074785 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.678922:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dce00 with x1855007616862976 ext(3260416->3379199) 00010000:00010000:1.0:1769074774.678934:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 28 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.678952:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 28 pid: 27843 timeout: 1862 lvb_type: 0 00002000:00010000:1.0:1769074774.678961:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b54d1dce00 00000100:00100000:0.0:1769074774.688344:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.689364:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616863104 00000100:00100000:2.0:1769074774.689377:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616863104 00000100:00100000:2.0:1769074774.689983:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14116 00000100:00100000:2.0:1769074774.689996:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62901:x1855007616863104:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074774.690017:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.690037:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdce expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.690073:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074774.690081:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.690094:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.690108:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773bc00, rc 0) 00000100:00100000:2.0:1769074774.693897:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62901:x1855007616863104:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3884us (5545us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.693916:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14116 00000100:00100000:3.0:1769074774.694190:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.696864:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616863232 00000100:00100000:1.0:1769074774.696881:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616863232 00010000:00010000:1.0:1769074774.696935:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 29 pid: 27843 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.696997:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7054 00000100:00100000:1.0:1769074774.697006:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616863232:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074774.697023:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074774.697043:0:36300:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074772 -> 1769074774 00002000:00010000:1.0:1769074774.697046:0:36300:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074772 -> 1769074774 00010000:00010000:1.0:1769074774.697052:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 29 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.697066:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 29 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.697103:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd53ced lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3260416->3379199) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb2 expref: 29 pid: 27843 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.697120:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.697122:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.697129:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.697138:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.697146:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.697148:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.697151:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.697163:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.697199:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.697212:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ece00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625150080:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074774.697333:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 19 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.697342:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625150208:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074774.697365:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074774.707311:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ece00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625150080:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074774.707327:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625150208:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074774.707353:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074774.707414:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616863232:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10407us (13223us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.707427:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7054 00000100:00100000:3.0:1769074774.725674:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.725719:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.725856:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616863360 00000100:00100000:1.0:1769074774.725867:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616863360 00002000:00100000:1.0:1769074774.725917:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d1ddf80 x1855007616863360/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.725929:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1ddf80 with x1855007616863360 ext(2097152->4194303) 00002000:00010000:1.0:1769074774.725936:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1ddf80 00000100:00100000:1.0:1769074774.725965:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616863488 00000100:00100000:1.0:1769074774.725969:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616863488 00002000:00100000:1.0:1769074774.725985:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d1dd880 x1855007616863488/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.725993:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dd880 with x1855007616863488 ext(2097152->4194303) 00002000:00010000:1.0:1769074774.725998:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1dd880 00000100:00100000:1.0:1769074774.726018:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8594 00000100:00100000:1.0:1769074774.726028:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616863360:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074774.727340:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8595 00000100:00100000:2.0:1769074774.727359:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616863488:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074774.736456:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.739397:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616863616 00000100:00100000:0.0:1769074774.739417:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616863616 00002000:00100000:0.0:1769074774.739497:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67fb97480 x1855007616863616/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.739742:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb97480 with x1855007616863616 ext(3145728->4194303) 00002000:00010000:0.0:1769074774.739749:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fb97480 00000100:00100000:0.0:1769074774.739800:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8596 00000100:00100000:0.0:1769074774.739849:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2409:x1855007616863616:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074774.778527:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.778784:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616863360:12345-192.168.204.60@tcp:3:cat.0 Request processed in 52754us (53120us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074774.778799:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8594 00002000:00100000:0.0:1769074774.778842:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d1ddf80 x1855007616863360/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074774.778853:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1ddf80 with x1855007616863360 ext(2097152->4194303) 00002000:00010000:0.0:1769074774.778860:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1ddf80 00000100:00100000:1.0:1769074774.780247:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616863872 00000100:00100000:1.0:1769074774.780258:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616863872 00000100:00100000:1.0:1769074774.780320:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13326 00000100:00100000:1.0:1769074774.780328:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+233:62913:x1855007616863872:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074774.780354:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074774.780378:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53d10 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdd5 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.780444:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780457:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780468:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780481:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780492:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780498:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780608:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd53d25 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780629:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd53d25 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780636:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd53d25 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074774.780646:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074774.780715:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780722:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780728:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780734:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780740:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074774.780743:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780750:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780765:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074774.780769:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074774.780773:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780779:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53d17 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780791:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780796:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780801:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780870:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea5400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074774.780873:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780878:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780885:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615300) 00010000:00010000:1.0:1769074774.780888:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074774.780890:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780896:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53d1e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074774.780929:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53d10 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdd5 expref: 236 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.780940:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd53d25 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdd5 expref: 235 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.780947:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074774.780950:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074774.780952:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f71400, rc 0) 00000100:00100000:1.0:1769074774.780998:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62913:x1855007616863872:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 671us (2484us total) trans 8589943959 rc 0/0 00000100:00100000:1.0:1769074774.781005:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13326 00000100:00100000:1.0:1769074774.785070:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616863488:12345-192.168.204.60@tcp:3:cat.0 Request processed in 57706us (59339us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074774.785086:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8595 00002000:00100000:1.0:1769074774.785098:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54d1dd880 x1855007616863488/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.785109:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dd880 with x1855007616863488 ext(2097152->4194303) 00002000:00010000:1.0:1769074774.785114:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1dd880 00000100:00100000:2.0:1769074774.792376:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2409:x1855007616863616:12345-192.168.204.60@tcp:3:cat.0 Request processed in 52552us (55917us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074774.792393:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8596 00002000:00100000:2.0:1769074774.792410:0:24028:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67fb97480 x1855007616863616/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.792421:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb97480 with x1855007616863616 ext(3145728->4194303) 00002000:00010000:2.0:1769074774.792428:0:24028:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fb97480 00000100:00100000:3.0:1769074774.800234:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.800322:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.800917:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864000 00000100:00100000:1.0:1769074774.800932:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864000 00000100:00100000:1.0:1769074774.801041:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1016 00000100:00100000:1.0:1769074774.801057:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:2408:x1855007616864000:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074774.801083:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074774.801125:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71400/0x7fe2323adfd53d25 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdd5 expref: 235 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.801154:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074774.801160:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074774.801236:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:2408:x1855007616864000:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 184us (1014us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.801244:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1016 00000100:00100000:2.0:1769074774.801348:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864256 00000100:00100000:2.0:1769074774.801354:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864256 00000100:00100000:2.0:1769074774.801401:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4674 00000100:00100000:2.0:1769074774.801410:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+234:62913:x1855007616864256:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074774.801682:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62913:x1855007616864256:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 274us (1361us total) trans 8589943960 rc 0/0 00000100:00100000:2.0:1769074774.801689:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4674 00000100:00100000:3.0:1769074774.826053:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074774.826098:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.826299:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864384 00000100:00100000:1.0:1769074774.826310:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864384 00002000:00100000:1.0:1769074774.826358:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b542fa9180 x1855007616864384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.826369:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9180 with x1855007616864384 ext(4194304->4808703) 00002000:00010000:1.0:1769074774.826375:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa9180 00000100:00100000:1.0:1769074774.826408:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864512 00000100:00100000:1.0:1769074774.826411:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864512 00002000:00100000:1.0:1769074774.826425:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b542fa9c00 x1855007616864512/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.826432:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9c00 with x1855007616864512 ext(4194304->4808703) 00002000:00010000:1.0:1769074774.826435:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa9c00 00000100:00100000:1.0:1769074774.826459:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8597 00000100:00100000:1.0:1769074774.826467:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2408:x1855007616864384:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074774.827227:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8598 00000100:00100000:0.0:1769074774.827239:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616864512:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074774.849656:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2408:x1855007616864384:12345-192.168.204.60@tcp:3:cat.0 Request processed in 23177us (23601us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074774.849670:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8597 00002000:00100000:1.0:1769074774.849683:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b542fa9180 x1855007616864384/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.849698:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9180 with x1855007616864384 ext(4194304->4808703) 00002000:00010000:1.0:1769074774.849704:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa9180 00000100:00100000:1.0:1769074774.856166:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616864512:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 28914us (30060us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074774.856182:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8598 00002000:00100000:1.0:1769074774.856196:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b542fa9c00 x1855007616864512/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/440 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 614400/614400 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.856206:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa9c00 with x1855007616864512 ext(4194304->4808703) 00002000:00010000:1.0:1769074774.856212:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa9c00 00000100:00100000:3.0:1769074774.869396:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.869853:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864640 00000100:00100000:2.0:1769074774.869866:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864640 00000100:00100000:2.0:1769074774.869940:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14117 00000100:00100000:2.0:1769074774.869951:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62913:x1855007616864640:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074774.869970:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.869989:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bddc expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.870026:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.870040:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.870047:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.870049:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.870053:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2703360->2863103] (req 2703360->2863103) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.870099:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.870113:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 24 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.870128:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150336:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.870212:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.870221:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.870230:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150464:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.870254:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.881066:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150336:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.881084:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150464:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.881131:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2703360->2863103] (req 2703360->2863103) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.881151:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b400, rc 0) 00000100:00100000:2.0:1769074774.881227:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62913:x1855007616864640:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11279us (11843us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.881236:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14117 00000100:00100000:0.0:1769074774.898340:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.898634:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864768 00000100:00100000:1.0:1769074774.898644:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864768 00010000:00010000:1.0:1769074774.898683:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.898725:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7055 00000100:00100000:1.0:1769074774.898734:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2409:x1855007616864768:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074774.898749:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074774.898772:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.898784:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.898846:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd53cfb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2453504->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc0 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.898864:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074774.898924:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2409:x1855007616864768:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 191us (595us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.898931:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7055 00000100:00100000:3.0:1769074774.914938:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.915049:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616864896 00000100:00100000:0.0:1769074774.915060:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616864896 00010000:00010000:0.0:1769074774.915112:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:0.0:1769074774.915155:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7056 00000100:00100000:0.0:1769074774.915165:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616864896:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074774.915183:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074774.915207:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.915225:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.915250:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd53cf4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdb9 expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074774.915267:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.915280:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074774.915287:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.915298:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074774.915336:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074774.915350:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625150592:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074774.915436:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074774.926028:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625150592:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074774.926061:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074774.926142:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616864896:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10977us (11213us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074774.926153:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7056 00000100:00100000:3.0:1769074774.927903:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.927996:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865024 00000100:00100000:1.0:1769074774.928006:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865024 00002000:00100000:1.0:1769074774.928056:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2703360->2707455) req@ffff89b542fab800 x1855007616865024/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.928067:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fab800 with x1855007616865024 ext(2703360->2707455) 00002000:00010000:1.0:1769074774.928074:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fab800 00000100:00100000:1.0:1769074774.928114:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8599 00000100:00100000:1.0:1769074774.928123:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616865024:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074774.928306:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616865024:12345-192.168.204.60@tcp:3:cat.0 Request processed in 184us (413us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074774.928314:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8599 00002000:00100000:1.0:1769074774.928321:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2703360->2707455) req@ffff89b542fab800 x1855007616865024/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074774.928328:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fab800 with x1855007616865024 ext(2703360->2707455) 00002000:00010000:1.0:1769074774.928331:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fab800 00000100:00100000:3.0:1769074774.939591:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.940129:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865152 00000100:00100000:2.0:1769074774.940143:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865152 00002000:00100000:2.0:1769074774.940209:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2859008->2863103) req@ffff89b542fa8a80 x1855007616865152/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/0 e 0 to 0 dl 1769074785 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.940221:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8a80 with x1855007616865152 ext(2859008->2863103) 00002000:00010000:2.0:1769074774.940228:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa8a80 00000100:00100000:2.0:1769074774.940279:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8600 00000100:00100000:2.0:1769074774.940289:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616865152:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074774.940498:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616865152:12345-192.168.204.60@tcp:3:cat.0 Request processed in 209us (919us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074774.940506:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8600 00002000:00100000:2.0:1769074774.940513:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2859008->2863103) req@ffff89b542fa8a80 x1855007616865152/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:310/0 lens 488/4536 e 0 to 0 dl 1769074785 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074774.940520:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b542fa8a80 with x1855007616865152 ext(2859008->2863103) 00002000:00010000:2.0:1769074774.940523:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b542fa8a80 00000100:00100000:1.0:1769074774.941856:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625150720:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074774.941916:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074774.942186:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625150720 00000100:00100000:3.0:1769074774.942193:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625150720 00000100:00100000:3.0:1769074774.942244:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1167 00000100:00100000:3.0:1769074774.942253:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625150720:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074774.942284:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1096873280 pending 3870720 free 3900067840 avail 3668545536 00000100:00100000:3.0:1769074774.942338:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625150720:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074774.942346:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1167 00000100:00100000:1.0:1769074774.942436:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625150720:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074774.982424:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074774.983264:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865280 00000100:00100000:2.0:1769074774.983285:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865280 00000100:00100000:2.0:1769074774.985321:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14118 00000100:00100000:2.0:1769074774.985341:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62911:x1855007616865280:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074774.985360:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074774.985380:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bde3 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.985412:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.985421:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.985428:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.985431:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074774.985434:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1638400->2097151] (req 1638400->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.985486:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.985900:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.985917:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150848:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.986029:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.986043:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 24 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074774.986052:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150976:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.986075:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074774.989257:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150848:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074774.989288:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625150976:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074774.989324:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1638400->2097151] (req 1638400->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074774.989347:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b800, rc 0) 00000100:00100000:2.0:1769074774.989414:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+25:62911:x1855007616865280:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 4081us (6999us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074774.989423:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14118 00000100:00100000:0.0:1769074774.996204:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074774.996255:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074774.997205:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865408 00000100:00100000:1.0:1769074774.997220:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865408 00010000:00010000:1.0:1769074774.997262:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.997322:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865536 00000100:00100000:1.0:1769074774.997325:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865536 00010000:00010000:1.0:1769074774.997342:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:1.0:1769074774.997473:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7057 00000100:00100000:1.0:1769074774.997482:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2408:x1855007616865408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074774.997511:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074774.997532:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.997546:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.997568:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53ce6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdab expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.997590:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074774.997671:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2408:x1855007616865408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 190us (1482us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074774.997679:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7057 00000100:00100000:1.0:1769074774.997696:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7058 00000100:00100000:1.0:1769074774.997703:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616865536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074774.997710:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074774.997718:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.997728:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.997740:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd53cbc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd9d expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074774.997747:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.997748:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074774.997754:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.997761:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074774.997791:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074774.997800:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ece00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625151104:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074774.997880:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.003078:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ece00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625151104:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074775.003118:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074775.004761:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616865536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6930us (8377us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.004776:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7058 00000100:00100000:1.0:1769074775.005003:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.006180:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865664 00000100:00100000:2.0:1769074775.006193:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865664 00002000:00100000:2.0:1769074775.006283:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1638400->1642495) req@ffff89b5460ef800 x1855007616865664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.006294:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef800 with x1855007616865664 ext(1638400->1642495) 00002000:00010000:2.0:1769074775.006301:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5460ef800 00000100:00100000:2.0:1769074775.006326:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8601 00000100:00100000:2.0:1769074775.006337:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2408:x1855007616865664:12345-192.168.204.60@tcp:3:multiop.0 00000100:00080000:1.0:1769074775.006343:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074775.006521:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+24:2408:x1855007616865664:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 187us (1519us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074775.006529:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8601 00002000:00100000:2.0:1769074775.006536:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1638400->1642495) req@ffff89b5460ef800 x1855007616865664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.006543:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef800 with x1855007616865664 ext(1638400->1642495) 00002000:00010000:2.0:1769074775.006547:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5460ef800 00000100:00100000:3.0:1769074775.019894:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.020845:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865792 00000100:00100000:2.0:1769074775.020856:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865792 00000100:00100000:2.0:1769074775.020921:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14119 00000100:00100000:2.0:1769074775.020932:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+24:62911:x1855007616865792:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074775.020949:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.020970:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdea expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.020999:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.021008:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.021010:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.021014:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->2572287] (req 2097152->2572287) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.021047:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 23 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.021065:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 23 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074775.021077:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625151232:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.021162:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074775.033558:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.033704:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625151232:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.033739:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->2572287] (req 2097152->2572287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.033758:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ac00, rc 0) 00000100:00100000:2.0:1769074775.033864:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+25:62911:x1855007616865792:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12933us (13978us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.033874:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14119 00000100:00100000:3.0:1769074775.035443:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616865920 00000100:00100000:3.0:1769074775.035456:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616865920 00002000:00100000:3.0:1769074775.035532:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2703360->2863103) req@ffff89b5460ef100 x1855007616865920/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.035544:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef100 with x1855007616865920 ext(2703360->2863103) 00010000:00010000:3.0:1769074775.035554:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 24 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:3.0:1769074775.035570:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 24 pid: 36516 timeout: 1862 lvb_type: 0 00002000:00010000:3.0:1769074775.035578:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5460ef100 00000100:00100000:3.0:1769074775.035622:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 785 00000100:00100000:3.0:1769074775.035633:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616865920:12345-192.168.204.60@tcp:4:cat.0 00000100:00080000:1.0:1769074775.036472:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074775.054045:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.054906:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616865920:12345-192.168.204.60@tcp:4:cat.0 Request processed in 19269us (21351us total) trans 17179874518 rc 0/0 00000100:00100000:2.0:1769074775.054910:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616866176 00000100:00100000:0.0:1769074775.054920:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 785 00000100:00100000:2.0:1769074775.054920:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616866176 00002000:00100000:0.0:1769074775.054933:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2703360->2863103) req@ffff89b5460ef100 x1855007616865920/t17179874518(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/448 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.054956:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef100 with x1855007616865920 ext(2703360->2863103) 00010000:00010000:0.0:1769074775.054967:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:0.0:1769074775.054991:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00002000:00010000:0.0:1769074775.054999:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5460ef100 00000100:00100000:2.0:1769074775.055024:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13327 00000100:00100000:2.0:1769074775.055033:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+235:62919:x1855007616866176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.055057:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.055098:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d48 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdf1 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.055199:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055218:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055232:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055249:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055255:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055262:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055379:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53d5d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055391:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53d5d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055398:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53d5d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074775.055407:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074775.055478:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055485:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055491:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055498:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055504:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773be00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.055508:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055516:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055528:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074775.055531:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.055536:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055542:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d4f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055557:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055563:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055568:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055573:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773b000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.055576:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055582:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055588:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4300) 00010000:00010000:2.0:1769074775.055591:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.055593:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055602:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d56 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.055645:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d48 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdf1 expref: 238 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.055657:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53d5d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdf1 expref: 237 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.055664:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074775.055666:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074775.055668:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773a000, rc 0) 00000100:00100000:2.0:1769074775.055720:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62919:x1855007616866176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 687us (1685us total) trans 8589943961 rc 0/0 00000100:00100000:2.0:1769074775.055727:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13327 00000100:00100000:1.0:1769074775.059462:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.059562:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.059595:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.059732:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616866560 00000100:00100000:2.0:1769074775.059745:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616866560 00000100:00100000:2.0:1769074775.059835:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616866688 00000100:00100000:2.0:1769074775.059839:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616866688 00010000:00010000:2.0:1769074775.059857:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074775.059881:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7059 00000100:00100000:2.0:1769074775.059891:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616866688:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.059908:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074775.059932:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074775.059952:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074775.059979:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53d33 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2703360->2863103) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bddc expref: 26 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074775.059995:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.059998:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.060004:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.060015:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.060054:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.060080:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625151360:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.060339:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.060409:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1017 00000100:00100000:3.0:1769074775.060420:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:2408:x1855007616866560:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074775.060429:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074775.060445:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53d5d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdf1 expref: 237 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.060460:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074775.060465:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074775.060493:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:2408:x1855007616866560:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 75us (930us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.060500:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1017 00000100:00100000:2.0:1769074775.064548:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625151360:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.064586:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.064668:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616866688:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4777us (5071us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.064680:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7059 00000100:00100000:0.0:1769074775.065816:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616866432 00000100:00100000:0.0:1769074775.065827:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616866432 00000100:00100000:0.0:1769074775.065882:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4675 00000100:00100000:0.0:1769074775.065891:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+236:62919:x1855007616866432:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074775.071236:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62919:x1855007616866432:12345-192.168.204.60@tcp:35:cat.0 Request processed in 5334us (11775us total) trans 8589943962 rc 0/0 00000100:00100000:1.0:1769074775.071251:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4675 00000100:00100000:3.0:1769074775.084846:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.085442:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616866816 00000100:00100000:0.0:1769074775.085454:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616866816 00002000:00100000:0.0:1769074775.085519:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2568192->2572287) req@ffff89b54cf68700 x1855007616866816/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.085530:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68700 with x1855007616866816 ext(2568192->2572287) 00002000:00010000:0.0:1769074775.085537:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf68700 00000100:00100000:0.0:1769074775.085575:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8602 00000100:00100000:0.0:1769074775.085583:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2409:x1855007616866816:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074775.085761:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2409:x1855007616866816:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 177us (964us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074775.085769:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8602 00002000:00100000:0.0:1769074775.085776:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2568192->2572287) req@ffff89b54cf68700 x1855007616866816/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.085783:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68700 with x1855007616866816 ext(2568192->2572287) 00002000:00010000:0.0:1769074775.085786:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf68700 00000100:00100000:2.0:1769074775.092302:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.092421:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.096581:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616867072 00000100:00100000:1.0:1769074775.096598:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616867072 00000100:00100000:1.0:1769074775.096838:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616866944 00000100:00100000:1.0:1769074775.096843:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616866944 00000100:00100000:1.0:1769074775.096924:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14120 00000100:00100000:1.0:1769074775.096934:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616867072:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074775.096954:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074775.096977:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53d6b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdff expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074775.097046:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625151488:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074775.097350:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.097509:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14121 00000100:00100000:3.0:1769074775.097520:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616866944:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074775.097529:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074775.097540:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53d72 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bdf8 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074775.097571:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625151616:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074775.097600:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00002000:00010000:3.0:1769074775.102908:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074773 -> 1769074774 00000100:00100000:3.0:1769074775.102940:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf6bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625151616:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074775.102990:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53d72 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdf8 expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074775.103012:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53d72 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdf8 expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074775.103026:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640031200, rc 0) 00002000:00010000:2.0:1769074775.103255:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074773 -> 1769074774 00000100:00100000:2.0:1769074775.103278:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625151488:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074775.103298:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53d6b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdff expref: 27 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.103309:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f70400/0x7fe2323adfd53d6b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdff expref: 27 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.103317:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70400, rc 0) 00000100:00100000:3.0:1769074775.103363:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616866944:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5843us (10940us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074775.103372:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14121 00000100:00100000:2.0:1769074775.107260:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616867072:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10313us (14954us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.107286:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14120 00000100:00100000:3.0:1769074775.133943:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.135119:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616867328 00000100:00100000:2.0:1769074775.135133:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616867328 00000100:00100000:2.0:1769074775.135209:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13328 00000100:00100000:2.0:1769074775.135219:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+237:62921:x1855007616867328:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074775.135246:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.135280:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53d79 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be06 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.135375:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135393:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135409:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135427:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135433:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135438:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135560:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d8e lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135571:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d8e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135577:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d8e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074775.135588:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074775.135874:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135882:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135887:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135894:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135899:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773ae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.135903:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135910:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135922:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074775.135924:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.135929:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135935:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53d80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135948:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135954:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135960:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135965:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773b000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.135967:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135973:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135980:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4300) 00010000:00010000:2.0:1769074775.135982:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.135985:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.135991:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd53d87 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.136030:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53d79 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be06 expref: 240 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.136044:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d8e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be06 expref: 239 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.136051:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074775.136053:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074775.136056:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773be00, rc 0) 00000100:00100000:2.0:1769074775.136234:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:62921:x1855007616867328:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1015us (2302us total) trans 8589943963 rc 0/0 00000100:00100000:2.0:1769074775.136243:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13328 00000100:00100000:3.0:1769074775.137659:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.137729:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616867456 00000100:00100000:2.0:1769074775.137738:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616867456 00000100:00100000:2.0:1769074775.137792:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14122 00000100:00100000:2.0:1769074775.137803:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:62919:x1855007616867456:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.137848:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.137866:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be0d expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.137892:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.137900:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.137901:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.137905:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.137943:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 24 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.137957:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 24 pid: 36516 timeout: 1862 lvb_type: 0 00000100:00100000:2.0:1769074775.137971:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625151744:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.138051:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.145777:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625151744:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.146147:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.146170:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773aa00, rc 0) 00000100:00100000:2.0:1769074775.146299:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62919:x1855007616867456:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8496us (8645us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.146310:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14122 00000100:00100000:0.0:1769074775.148187:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.148236:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.148260:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.148465:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616867584 00000100:00100000:2.0:1769074775.148475:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616867584 00002000:00100000:2.0:1769074775.148537:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1638400->2097151) req@ffff89b54ce9c700 x1855007616867584/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.148547:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9c700 with x1855007616867584 ext(1638400->2097151) 00010000:00010000:2.0:1769074775.148556:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:2.0:1769074775.148574:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 25 pid: 36516 timeout: 1862 lvb_type: 0 00002000:00010000:2.0:1769074775.148581:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9c700 00000100:00100000:2.0:1769074775.148611:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 786 00000100:00100000:2.0:1769074775.148620:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+25:2408:x1855007616867584:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:1.0:1769074775.149845:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616867840 00000100:00100000:1.0:1769074775.149857:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616867840 00000100:00100000:1.0:1769074775.149914:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4676 00000100:00100000:1.0:1769074775.149923:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:62921:x1855007616867840:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074775.150143:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5446c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+240:62921:x1855007616867840:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 219us (1905us total) trans 8589943964 rc 0/0 00000100:00100000:1.0:1769074775.150151:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4676 00000100:00100000:3.0:1769074775.152731:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616867968 00000100:00100000:3.0:1769074775.152752:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616867968 00000100:00100000:2.0:1769074775.153201:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1018 00000100:00100000:2.0:1769074775.153216:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+240:2407:x1855007616867968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.153235:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074775.153264:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53d8e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be06 expref: 240 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.153297:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074775.153303:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.153367:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:2407:x1855007616867968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 156us (5105us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.153376:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1018 00000100:00100000:3.0:1769074775.160538:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.161010:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868096 00000100:00100000:2.0:1769074775.161023:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868096 00000100:00100000:2.0:1769074775.161093:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14123 00000100:00100000:2.0:1769074775.161104:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+27:62925:x1855007616868096:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.161122:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.161143:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be14 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.161178:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.161183:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.161195:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.161208:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b400, rc 0) 00000100:00100000:2.0:1769074775.161263:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+28:62925:x1855007616868096:12345-192.168.204.60@tcp:101:cat.0 Request processed in 161us (738us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.161303:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14123 00000100:00100000:1.0:1769074775.168553:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.168660:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868224 00000100:00100000:3.0:1769074775.168672:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868224 00000100:00100000:3.0:1769074775.168726:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14124 00000100:00100000:3.0:1769074775.168736:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62921:x1855007616868224:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074775.168757:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074775.168782:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be1b expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168811:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074775.168829:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168839:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168845:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 3 00010000:00010000:3.0:1769074775.168849:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1970176->1978367] (req 1970176->1978367) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168889:0:27843:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168903:0:27843:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168916:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [1970176->1978367] (req 1970176->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.168928:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b640031200, rc 0) 00000100:00100000:3.0:1769074775.169107:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62921:x1855007616868224:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 372us (568us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.169115:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14124 00000100:00100000:1.0:1769074775.174280:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2408:x1855007616867584:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 25636us (26080us total) trans 17179874519 rc 0/0 00000100:00100000:1.0:1769074775.174296:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 786 00002000:00100000:1.0:1769074775.174312:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1638400->2097151) req@ffff89b54ce9c700 x1855007616867584/t17179874519(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/448 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074775.174322:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9c700 with x1855007616867584 ext(1638400->2097151) 00010000:00010000:1.0:1769074775.174333:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 28 pid: 36516 timeout: 1862 lvb_type: 0 00010000:00010000:1.0:1769074775.174348:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:1.0:1769074775.174356:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9c700 00000100:00100000:1.0:1769074775.177478:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.177642:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868352 00000100:00100000:3.0:1769074775.177652:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868352 00010000:00010000:3.0:1769074775.177690:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.177732:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7060 00000100:00100000:3.0:1769074775.177741:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2408:x1855007616868352:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074775.177756:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074775.177778:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.177791:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.177837:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b800/0x7fe2323adfd53d3a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1638400->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bde3 expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.177850:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.177852:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.177861:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.177869:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.177877:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.177878:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.177881:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.177891:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.177927:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.177939:0:6500:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.177953:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625152128:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074775.178032:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.178040:0:6500:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 26 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.178049:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625152256:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074775.178069:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b642c51180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.185890:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625152128:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.185906:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625152256:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.185930:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.186012:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+27:2408:x1855007616868352:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8270us (8545us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.186024:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7060 00000100:00100000:1.0:1769074775.193540:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.193587:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.193761:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868480 00000100:00100000:2.0:1769074775.193773:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868480 00002000:00100000:2.0:1769074775.193868:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5460efb80 x1855007616868480/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.193879:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460efb80 with x1855007616868480 ext(1048576->2097151) 00010000:00010000:2.0:1769074775.193888:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.193906:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074775.193914:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5460efb80 00000100:00100000:2.0:1769074775.193941:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868608 00000100:00100000:2.0:1769074775.193944:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868608 00002000:00100000:2.0:1769074775.193959:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546c68380 x1855007616868608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.193967:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c68380 with x1855007616868608 ext(1048576->2097151) 00010000:00010000:2.0:1769074775.193983:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.193992:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074775.193998:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546c68380 00000100:00100000:2.0:1769074775.194015:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 787 00000100:00100000:2.0:1769074775.194024:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616868480:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074775.194831:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 788 00000100:00100000:0.0:1769074775.194841:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+27:2408:x1855007616868608:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074775.208154:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616868480:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 14118us (14612us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074775.208171:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 787 00002000:00100000:0.0:1769074775.208184:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5460efb80 x1855007616868480/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.208195:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460efb80 with x1855007616868480 ext(1048576->2097151) 00010000:00010000:0.0:1769074775.208205:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.208221:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:0.0:1769074775.208234:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5460efb80 00000100:00100000:0.0:1769074775.212997:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+27:2408:x1855007616868608:12345-192.168.204.60@tcp:3:cat.0 Request processed in 18131us (19382us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074775.213014:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 788 00002000:00100000:0.0:1769074775.213028:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b546c68380 x1855007616868608/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.213038:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546c68380 with x1855007616868608 ext(1048576->2097151) 00010000:00010000:0.0:1769074775.213049:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.213065:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:0.0:1769074775.213071:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546c68380 00000100:00100000:1.0:1769074775.231128:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.231223:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.231260:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.231422:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868864 00000100:00100000:3.0:1769074775.231434:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868864 00010000:00010000:3.0:1769074775.231477:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.231519:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868992 00000100:00100000:3.0:1769074775.231521:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868992 00010000:00010000:3.0:1769074775.231532:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.231559:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7061 00000100:00100000:3.0:1769074775.231567:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+27:2407:x1855007616868864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074775.231581:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074775.231607:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.231628:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.231657:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53da3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be14 expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.231677:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074775.231734:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2407:x1855007616868864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 166us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.231741:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7061 00000100:00100000:3.0:1769074775.231795:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7062 00000100:00100000:3.0:1769074775.231801:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616868992:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074775.231847:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074775.231855:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.231866:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.231880:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53d9c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be0d expref: 27 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.231889:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.231891:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.231899:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.231909:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.231943:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074775.231957:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625152384:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074775.232010:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.238363:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616868736 00000100:00100000:2.0:1769074775.238380:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616868736 00000100:00100000:2.0:1769074775.239076:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14125 00000100:00100000:2.0:1769074775.239102:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62919:x1855007616868736:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.239122:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.239142:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be22 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.239178:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.239186:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.239188:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.239192:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.239228:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.239242:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 25 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.239255:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625152512:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.239363:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.240007:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.240114:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625152384:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074775.240146:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074775.240197:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616868992:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8393us (8934us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.240206:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7062 00000100:00100000:0.0:1769074775.240255:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869120 00000100:00100000:0.0:1769074775.240272:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869120 00002000:00100000:0.0:1769074775.240312:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1970176->1974271) req@ffff89b54cf68a80 x1855007616869120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.240323:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68a80 with x1855007616869120 ext(1970176->1974271) 00002000:00010000:0.0:1769074775.240329:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf68a80 00000100:00100000:0.0:1769074775.240351:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8603 00000100:00100000:0.0:1769074775.240359:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616869120:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074775.240513:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616869120:12345-192.168.204.60@tcp:3:cat.0 Request processed in 154us (511us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074775.240520:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8603 00002000:00100000:0.0:1769074775.240526:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1970176->1974271) req@ffff89b54cf68a80 x1855007616869120/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.240534:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68a80 with x1855007616869120 ext(1970176->1974271) 00002000:00010000:0.0:1769074775.240538:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf68a80 00000100:00100000:0.0:1769074775.243369:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.243547:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869248 00000100:00100000:2.0:1769074775.243558:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869248 00002000:00100000:2.0:1769074775.243611:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2572288->4194303) req@ffff89b54d1de300 x1855007616869248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.243622:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1de300 with x1855007616869248 ext(2572288->4194303) 00010000:00010000:2.0:1769074775.243631:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 26 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.243647:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 26 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074775.243655:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d1de300 00000100:00100000:2.0:1769074775.243691:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 789 00000100:00100000:2.0:1769074775.243700:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+26:2409:x1855007616869248:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074775.244850:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.244882:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.244905:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869376 00000100:00100000:3.0:1769074775.244914:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869376 00002000:00100000:3.0:1769074775.244954:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1974272->1978367) req@ffff89b54cf68000 x1855007616869376/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.244966:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68000 with x1855007616869376 ext(1974272->1978367) 00002000:00010000:3.0:1769074775.244972:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf68000 00000100:00100000:3.0:1769074775.244992:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869504 00000100:00100000:3.0:1769074775.244996:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869504 00002000:00100000:3.0:1769074775.245012:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2572287) req@ffff89b54ce9e680 x1855007616869504/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.245020:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9e680 with x1855007616869504 ext(2097152->2572287) 00010000:00010000:3.0:1769074775.245028:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.245042:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.245050:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9e680 00000100:00100000:3.0:1769074775.245080:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 790 00000100:00100000:3.0:1769074775.245183:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2409:x1855007616869504:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074775.245278:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8604 00000100:00100000:1.0:1769074775.245289:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616869376:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074775.245412:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625152512:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074775.245413:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616869376:12345-192.168.204.60@tcp:3:cat.0 Request processed in 126us (569us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074775.245422:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8604 00002000:00100000:1.0:1769074775.245429:0:16436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2be:0x0] (1974272->1978367) req@ffff89b54cf68000 x1855007616869376/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074775.245437:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68000 with x1855007616869376 ext(1974272->1978367) 00002000:00010000:1.0:1769074775.245440:0:16436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54cf68000 00010000:00010000:3.0:1769074775.245446:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.245463:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773be00, rc 0) 00000100:00100000:3.0:1769074775.247925:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62919:x1855007616868736:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6397us (14374us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.247948:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14125 00000100:00100000:3.0:1769074775.263457:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2409:x1855007616869504:12345-192.168.204.60@tcp:4:cat.0 Request processed in 18263us (18561us total) trans 17179874520 rc 0/0 00000100:00100000:3.0:1769074775.263472:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 790 00002000:00100000:3.0:1769074775.263486:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2572287) req@ffff89b54ce9e680 x1855007616869504/t17179874520(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/448 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.263931:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9e680 with x1855007616869504 ext(2097152->2572287) 00010000:00010000:3.0:1769074775.263943:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.263961:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.263969:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54ce9e680 00000100:00100000:3.0:1769074775.265057:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2409:x1855007616869248:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21351us (21691us total) trans 0 rc 1622016/1622016 00000100:00100000:3.0:1769074775.265069:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 789 00002000:00100000:3.0:1769074775.265090:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2572288->4194303) req@ffff89b54d1de300 x1855007616869248/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 1622016/1622016 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.265099:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1de300 with x1855007616869248 ext(2572288->4194303) 00010000:00010000:3.0:1769074775.265108:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.265121:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 28 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.265127:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54d1de300 00000100:00100000:1.0:1769074775.279346:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.279506:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869632 00000100:00100000:3.0:1769074775.279519:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869632 00000100:00100000:3.0:1769074775.279581:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14126 00000100:00100000:3.0:1769074775.279592:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+28:62925:x1855007616869632:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074775.279623:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074775.279645:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be29 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.279675:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074775.279678:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074775.279683:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.279694:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.279705:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866200, rc 0) 00000100:00100000:3.0:1769074775.279765:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62925:x1855007616869632:12345-192.168.204.60@tcp:101:cat.0 Request processed in 176us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.279772:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14126 00000100:00100000:2.0:1769074775.283155:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.284257:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869760 00000100:00100000:0.0:1769074775.284281:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869760 00010000:00010000:0.0:1769074775.284345:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:0.0:1769074775.284455:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7063 00000100:00100000:0.0:1769074775.284465:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2407:x1855007616869760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074775.284481:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074775.284519:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.284536:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.284571:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53d41 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->2572287) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdea expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.284589:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074775.284592:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074775.284601:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.284622:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.284631:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074775.284639:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074775.284643:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.284649:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.284686:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074775.284702:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625152640:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074775.284877:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074775.284892:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625152768:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074775.284922:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074775.289290:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625152640:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.297053:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.297175:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616869888 00000100:00100000:3.0:1769074775.297186:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616869888 00002000:00100000:3.0:1769074775.297236:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67fb94e00 x1855007616869888/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.297247:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb94e00 with x1855007616869888 ext(3145728->4194303) 00002000:00010000:3.0:1769074775.297254:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fb94e00 00000100:00100000:3.0:1769074775.297302:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8605 00000100:00100000:3.0:1769074775.297312:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2409:x1855007616869888:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074775.300766:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625152768:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074775.300817:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074775.300975:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2407:x1855007616869760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16509us (17829us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.300988:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7063 00000100:00100000:3.0:1769074775.304955:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.305031:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870016 00000100:00100000:1.0:1769074775.305042:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870016 00002000:00100000:1.0:1769074775.305178:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54ce9f100 x1855007616870016/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074775.305191:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9f100 with x1855007616870016 ext(2097152->4194303) 00002000:00010000:1.0:1769074775.305199:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9f100 00000100:00100000:1.0:1769074775.305237:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8606 00000100:00100000:1.0:1769074775.305247:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616870016:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074775.386942:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616870016:12345-192.168.204.60@tcp:3:cat.0 Request processed in 81682us (81981us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074775.386957:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8606 00002000:00100000:1.0:1769074775.386971:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54ce9f100 x1855007616870016/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074775.386982:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9f100 with x1855007616870016 ext(2097152->4194303) 00002000:00010000:1.0:1769074775.386989:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9f100 00000100:00100000:2.0:1769074775.391138:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2409:x1855007616869888:12345-192.168.204.60@tcp:3:cat.0 Request processed in 93815us (94089us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074775.391153:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8605 00002000:00100000:2.0:1769074775.391166:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67fb94e00 x1855007616869888/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.391177:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fb94e00 with x1855007616869888 ext(3145728->4194303) 00002000:00010000:2.0:1769074775.391183:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fb94e00 00000100:00100000:3.0:1769074775.402509:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.406670:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870144 00000100:00100000:0.0:1769074775.406686:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870144 00000100:00100000:0.0:1769074775.406767:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14127 00000100:00100000:0.0:1769074775.406776:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+28:2406:x1855007616870144:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074775.406793:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074775.406854:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd53dbf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be30 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074775.406915:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625152896:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074775.407010:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.408087:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.408212:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870272 00000100:00100000:2.0:1769074775.408219:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870272 00000100:00100000:2.0:1769074775.408272:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14128 00000100:00100000:2.0:1769074775.408281:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62928:x1855007616870272:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074775.408294:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.408308:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be37 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.408332:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.408342:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 31 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.408349:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.408351:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.408356:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3641344->3956735] (req 3641344->3956735) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.408382:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 31 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.408396:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 31 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.408408:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625153024:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.408500:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 26 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.408510:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 26 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.408518:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625153152:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.408542:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00002000:00010000:0.0:1769074775.414581:0:27843:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074774 -> 1769074775 00000100:00100000:0.0:1769074775.414617:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625152896:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074775.415055:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd53dbf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be30 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.415163:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd53dbf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be30 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.415182:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e400, rc 0) 00000100:00100000:0.0:1769074775.415376:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2406:x1855007616870144:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8600us (12877us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074775.415386:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14127 00000100:00100000:2.0:1769074775.415951:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb96a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625153024:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074775.420580:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.420609:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.420735:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625153152:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.420773:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3641344->3956735] (req 3641344->3956735) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.420799:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b000, rc 0) 00000100:00100000:2.0:1769074775.420890:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62928:x1855007616870272:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12609us (12817us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.420899:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14128 00000100:00100000:2.0:1769074775.420930:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870528 00000100:00100000:2.0:1769074775.420937:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870528 00000100:00100000:2.0:1769074775.420994:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14129 00000100:00100000:2.0:1769074775.421004:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616870528:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.421020:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.421041:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53dcd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be3e expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.421063:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.421074:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53dcd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be3e expref: 28 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.421105:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53dcd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be3e expref: 28 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.421117:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773aa00, rc 0) 00000100:00100000:2.0:1769074775.421138:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616870528:12345-192.168.204.60@tcp:101:cat.0 Request processed in 135us (529us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.421144:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14129 00000100:00100000:0.0:1769074775.421776:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.422010:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870400 00000100:00100000:1.0:1769074775.422017:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870400 00010000:00010000:1.0:1769074775.422044:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 27 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:1.0:1769074775.422091:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870656 00000100:00100000:1.0:1769074775.422094:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870656 00010000:00010000:1.0:1769074775.422108:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 29 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:1.0:1769074775.422127:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7064 00000100:00100000:1.0:1769074775.422135:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616870400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074775.422146:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074775.422166:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 27 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.422171:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7065 00010000:00010000:1.0:1769074775.422176:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 27 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.422180:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2408:x1855007616870656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074775.422188:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074775.422197:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53db1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be22 expref: 27 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.422198:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 29 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.422208:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 29 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:1.0:1769074775.422597:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074775.422658:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616870400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 522us (2087us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.422666:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7064 00010000:00010000:3.0:1769074775.426911:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd53db8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be29 expref: 29 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.426938:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.426940:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074775.426948:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.426960:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.427156:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074775.427175:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625153280:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074775.427281:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5497b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.434884:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.435189:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616870912 00000100:00100000:2.0:1769074775.435200:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616870912 00000100:00100000:2.0:1769074775.435269:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13329 00000100:00100000:2.0:1769074775.435281:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+239:62930:x1855007616870912:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074775.435312:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.435336:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53dd4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be45 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.435420:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435434:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435446:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435461:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435469:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435475:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435608:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53de9 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435621:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53de9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435628:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53de9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074775.435638:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074775.435735:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435742:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435748:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435753:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435759:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773b800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.435764:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435772:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435784:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074775.435786:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.435792:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435798:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53ddb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435828:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435834:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435840:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435849:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773a000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.435853:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435859:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435866:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4300) 00010000:00010000:2.0:1769074775.435869:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.435872:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435876:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a000/0x7fe2323adfd53de2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074775.435921:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53dd4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be45 expref: 242 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.435937:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53de9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be45 expref: 241 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.435944:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074775.435946:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074775.435949:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b200, rc 0) 00000100:00100000:2.0:1769074775.436695:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62930:x1855007616870912:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1411us (1815us total) trans 8589943965 rc 0/0 00000100:00100000:2.0:1769074775.436705:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13329 00000100:00100000:1.0:1769074775.438415:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.438466:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.438574:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871040 00000100:00100000:3.0:1769074775.438584:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871040 00000100:00100000:0.0:1769074775.438598:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871168 00000100:00100000:0.0:1769074775.438603:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871168 00000100:00100000:3.0:1769074775.438652:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14130 00000100:00100000:3.0:1769074775.438661:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:62919:x1855007616871168:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074775.438682:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074775.438702:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be53 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.438727:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14131 00010000:00010000:3.0:1769074775.438733:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.438736:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+29:62925:x1855007616871040:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074775.438742:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.438744:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074775.438744:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074775.438748:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.438751:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be4c expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.438764:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.438768:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.438776:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.438781:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.438794:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:3.0:1769074775.438838:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625153408:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074775.438917:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5b0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1769074775.444288:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b600, rc 0) 00000100:00100000:2.0:1769074775.444375:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62925:x1855007616871040:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5635us (5964us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.444385:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14131 00000100:00100000:3.0:1769074775.448012:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625153408:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074775.448051:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074775.448071:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718600, rc 0) 00000100:00100000:3.0:1769074775.448152:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62919:x1855007616871168:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9491us (9684us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.448164:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14130 00000100:00100000:0.0:1769074775.448989:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625153280:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074775.449006:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074775.449053:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2408:x1855007616870656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 26873us (27279us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.449062:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7065 00000100:00100000:1.0:1769074775.452380:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.452428:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.452478:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.452641:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871680 00000100:00100000:3.0:1769074775.452652:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871680 00002000:00100000:3.0:1769074775.452703:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3641344->3645439) req@ffff89b5499d3800 x1855007616871680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.452715:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d3800 with x1855007616871680 ext(3641344->3645439) 00000100:00100000:2.0:1769074775.452718:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871424 00000100:00100000:2.0:1769074775.452723:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871424 00010000:00010000:3.0:1769074775.452725:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.452741:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.452748:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d3800 00000100:00100000:2.0:1769074775.452769:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4677 00000100:00100000:3.0:1769074775.452769:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 791 00000100:00100000:2.0:1769074775.452778:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62930:x1855007616871424:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074775.452778:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2408:x1855007616871680:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074775.452965:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2408:x1855007616871680:12345-192.168.204.60@tcp:3:cat.0 Request processed in 187us (485us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074775.452973:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 791 00002000:00100000:3.0:1769074775.452980:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3641344->3645439) req@ffff89b5499d3800 x1855007616871680/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.452989:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d3800 with x1855007616871680 ext(3641344->3645439) 00010000:00010000:3.0:1769074775.452994:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.453003:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:62930:x1855007616871424:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 226us (633us total) trans 8589943966 rc 0/0 00000100:00100000:2.0:1769074775.453010:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4677 00010000:00010000:3.0:1769074775.453011:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.453017:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d3800 00010000:00010000:2.0:1769074775.453071:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074775.453414:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871552 00000100:00100000:0.0:1769074775.453421:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871552 00000100:00100000:0.0:1769074775.453482:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1019 00000100:00100000:0.0:1769074775.453490:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+242:2406:x1855007616871552:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074775.453499:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074775.453521:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53de9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be45 expref: 242 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.453536:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074775.453540:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074775.453574:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:2406:x1855007616871552:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 85us (1144us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.453581:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1019 00000100:00100000:3.0:1769074775.460781:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.465214:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871808 00000100:00100000:1.0:1769074775.465230:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871808 00002000:00100000:1.0:1769074775.465304:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3952640->3956735) req@ffff89b545108e00 x1855007616871808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074775.465316:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108e00 with x1855007616871808 ext(3952640->3956735) 00010000:00010000:1.0:1769074775.465325:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:1.0:1769074775.465341:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:1.0:1769074775.465348:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b545108e00 00000100:00100000:1.0:1769074775.465392:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 792 00000100:00100000:1.0:1769074775.465401:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2409:x1855007616871808:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074775.465867:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2409:x1855007616871808:12345-192.168.204.60@tcp:3:cat.0 Request processed in 466us (5097us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074775.465877:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 792 00002000:00100000:1.0:1769074775.465884:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3952640->3956735) req@ffff89b545108e00 x1855007616871808/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074775.465891:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545108e00 with x1855007616871808 ext(3952640->3956735) 00010000:00010000:1.0:1769074775.465897:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:1.0:1769074775.465912:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:1.0:1769074775.465919:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b545108e00 00000100:00100000:1.0:1769074775.481234:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.481650:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616871936 00000100:00100000:3.0:1769074775.481661:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616871936 00002000:00100000:3.0:1769074775.481708:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3641344->3956735) req@ffff89b5499d0a80 x1855007616871936/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.481719:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0a80 with x1855007616871936 ext(3641344->3956735) 00010000:00010000:3.0:1769074775.481727:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.481744:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.481750:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d0a80 00000100:00100000:3.0:1769074775.481787:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 793 00000100:00100000:3.0:1769074775.481796:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+29:2409:x1855007616871936:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074775.507212:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.507257:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.507290:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872064 00000100:00100000:2.0:1769074775.507295:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.507301:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872064 00000100:00100000:2.0:1769074775.507330:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.507344:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.507351:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 331 00000100:00100000:0.0:1769074775.507360:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b08050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2407:x1855007616872064:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074775.507365:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.507379:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.507394:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.507466:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872320 00000100:00100000:1.0:1769074775.507470:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872320 00000100:00100000:1.0:1769074775.507501:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1448 00000100:00100000:1.0:1769074775.507507:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616872320:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074775.507621:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b08050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2407:x1855007616872064:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 261us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.507629:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 331 00000100:00100000:0.0:1769074775.507673:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873088 00000100:00100000:0.0:1769074775.507678:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873088 00000100:00100000:0.0:1769074775.507707:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872704 00000100:00100000:0.0:1769074775.507710:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872704 00000100:00100000:0.0:1769074775.507727:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1449 00000100:00100000:0.0:1769074775.507734:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2407:x1855007616873088:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074775.507804:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1450 00000100:00100000:3.0:1769074775.507846:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616872704:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074775.515299:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.515358:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872576 00000100:00100000:2.0:1769074775.515369:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872576 00000100:00100000:2.0:1769074775.515432:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872832 00000100:00100000:2.0:1769074775.515437:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872832 00000100:00100000:2.0:1769074775.515451:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872192 00000100:00100000:2.0:1769074775.515455:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872192 00000100:00100000:2.0:1769074775.515469:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872448 00000100:00100000:2.0:1769074775.515472:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872448 00000100:00100000:2.0:1769074775.515485:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616872960 00000100:00100000:2.0:1769074775.515498:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616872960 00000100:00100000:2.0:1769074775.515518:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2318 00000100:00100000:2.0:1769074775.515527:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2407:x1855007616872576:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074775.515697:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616872320:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 8185us (8435us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.515709:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1448 00000100:00080000:1.0:1769074775.515744:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074775.515749:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074775.515751:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074775.515766:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2319 00000100:00100000:1.0:1769074775.515774:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2407:x1855007616872832:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074775.515802:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2407:x1855007616872832:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (8471us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.515822:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2319 00000100:00100000:1.0:1769074775.515842:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2320 00000100:00100000:1.0:1769074775.515848:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616872192:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074775.515873:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616872192:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (8506us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.515882:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2320 00000100:00100000:1.0:1769074775.515894:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2321 00000100:00100000:1.0:1769074775.515900:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616872448:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074775.515925:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616872448:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (8545us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.515931:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2321 00000100:00100000:1.0:1769074775.515941:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2322 00000100:00100000:1.0:1769074775.515947:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616872960:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074775.515971:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616872960:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (683us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.515977:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2322 00000100:00100000:1.0:1769074775.516045:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.516064:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873216 00000100:00100000:1.0:1769074775.516068:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873216 00000100:00100000:1.0:1769074775.516573:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1451 00000100:00100000:1.0:1769074775.516581:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616873216:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074775.516715:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616873216:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 134us (669us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074775.516722:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1451 00000100:00080000:1.0:1769074775.516743:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074775.516745:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2407:x1855007616872576:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 1219us (9449us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.516753:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2318 00000100:00100000:0.0:1769074775.521433:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2407:x1855007616873088:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 13682us (14070us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.522044:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1449 00000100:00100000:3.0:1769074775.524663:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616872704:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 16805us (17253us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074775.524684:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1450 00000100:00100000:3.0:1769074775.529425:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2409:x1855007616871936:12345-192.168.204.60@tcp:4:cat.0 Request processed in 47613us (48187us total) trans 17179874521 rc 0/0 00000100:00100000:3.0:1769074775.529442:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 793 00002000:00100000:3.0:1769074775.529455:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3641344->3956735) req@ffff89b5499d0a80 x1855007616871936/t17179874521(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/448 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.529464:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0a80 with x1855007616871936 ext(3641344->3956735) 00010000:00010000:3.0:1769074775.529474:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:3.0:1769074775.529493:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 1863 lvb_type: 0 00002000:00010000:3.0:1769074775.529501:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d0a80 00000100:00100000:2.0:1769074775.533355:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.534040:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873344 00000100:00100000:3.0:1769074775.534055:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873344 00000100:00100000:2.0:1769074775.534341:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14132 00000100:00100000:2.0:1769074775.534352:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62930:x1855007616873344:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074775.534375:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.534399:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be5a expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.534549:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.534553:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.534560:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.534573:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 28 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.534585:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ba00, rc 0) 00000100:00100000:2.0:1769074775.534743:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62930:x1855007616873344:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 392us (1398us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.534751:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14132 00000100:00100000:1.0:1769074775.536571:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.536993:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873472 00000100:00100000:2.0:1769074775.537003:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873472 00010000:00010000:2.0:1769074775.537041:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.537191:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7066 00000100:00100000:2.0:1769074775.537203:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2409:x1855007616873472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.537218:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1769074775.537236:0:6500:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074774 -> 1769074775 00002000:00010000:2.0:1769074775.537238:0:6500:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074774 -> 1769074775 00010000:00010000:2.0:1769074775.537244:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.537280:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.537307:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b000/0x7fe2323adfd53dc6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3641344->3956735) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be37 expref: 31 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.537323:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.537325:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.537332:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.537342:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.537351:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.537353:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.537357:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.537365:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.537371:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.537373:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.537377:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.537388:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.537425:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.537712:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625153536:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.537910:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.537920:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625153664:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.537952:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.537961:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625153792:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.537984:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074775.547620:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.547714:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.547802:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873600 00000100:00100000:0.0:1769074775.547850:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873600 00002000:00100000:0.0:1769074775.547909:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5499d1c00 x1855007616873600/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.547921:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1c00 with x1855007616873600 ext(4194304->4808703) 00002000:00010000:0.0:1769074775.547928:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d1c00 00000100:00100000:0.0:1769074775.547948:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873728 00000100:00100000:0.0:1769074775.547951:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873728 00002000:00100000:0.0:1769074775.547966:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2392064->2396159) req@ffff89b5499d1f80 x1855007616873728/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.547973:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1f80 with x1855007616873728 ext(2392064->2396159) 00002000:00010000:0.0:1769074775.547975:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d1f80 00000100:00100000:0.0:1769074775.547996:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8607 00000100:00100000:0.0:1769074775.548004:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2408:x1855007616873600:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074775.548469:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8608 00000100:00100000:0.0:1769074775.548477:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616873728:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074775.548558:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616873728:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 82us (844us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074775.548566:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8608 00002000:00100000:0.0:1769074775.548573:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2392064->2396159) req@ffff89b5499d1f80 x1855007616873728/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.548582:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1f80 with x1855007616873728 ext(2392064->2396159) 00002000:00010000:0.0:1769074775.548586:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d1f80 00000100:00100000:2.0:1769074775.549407:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625153536:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.549423:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625153664:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.553880:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625153792:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.553916:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.553978:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2409:x1855007616873472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16776us (17413us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.553994:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7066 00000100:00100000:0.0:1769074775.559205:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.559313:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873856 00000100:00100000:3.0:1769074775.559325:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873856 00002000:00100000:3.0:1769074775.559373:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5461e6680 x1855007616873856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.559385:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e6680 with x1855007616873856 ext(4194304->4808703) 00002000:00010000:3.0:1769074775.559391:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e6680 00000100:00100000:3.0:1769074775.559431:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8609 00000100:00100000:3.0:1769074775.559440:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616873856:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074775.568798:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2408:x1855007616873600:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20781us (21186us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074775.568852:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8607 00002000:00100000:0.0:1769074775.568867:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5499d1c00 x1855007616873600/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.568878:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1c00 with x1855007616873600 ext(4194304->4808703) 00002000:00010000:0.0:1769074775.568884:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d1c00 00000100:00100000:3.0:1769074775.573105:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616873856:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 13653us (13903us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074775.573120:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8609 00002000:00100000:3.0:1769074775.573132:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5461e6680 x1855007616873856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 614400/614400 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.573142:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5461e6680 with x1855007616873856 ext(4194304->4808703) 00002000:00010000:3.0:1769074775.573148:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5461e6680 00000100:00100000:1.0:1769074775.574671:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.574770:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.579852:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874112 00000100:00100000:2.0:1769074775.579869:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874112 00000100:00100000:1.0:1769074775.580171:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.581506:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616873984 00000100:00100000:2.0:1769074775.581518:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616873984 00000100:00100000:0.0:1769074775.582432:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14133 00000100:00100000:0.0:1769074775.582449:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2409:x1855007616874112:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074775.582469:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074775.582488:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e0c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be68 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.582523:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e0c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be68 expref: 32 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.582540:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e0c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be68 expref: 32 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.582553:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f600, rc 0) 00000100:00100000:0.0:1769074775.582639:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2409:x1855007616874112:12345-192.168.204.60@tcp:101:cat.0 Request processed in 196us (7976us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074775.582646:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14133 00000100:00100000:0.0:1769074775.582669:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14134 00000100:00100000:0.0:1769074775.582676:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2409:x1855007616873984:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074775.582683:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074775.582693:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd53e13 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be61 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074775.582742:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625153920:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074775.582798:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59240 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.587564:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874240 00000100:00100000:3.0:1769074775.587588:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874240 00002000:00100000:3.0:1769074775.587654:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2879488->2883583) req@ffff89b67fa22680 x1855007616874240/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.587665:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22680 with x1855007616874240 ext(2879488->2883583) 00002000:00010000:3.0:1769074775.587671:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa22680 00000100:00100000:3.0:1769074775.587706:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8610 00000100:00100000:3.0:1769074775.587715:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616874240:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074775.587934:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616874240:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 219us (7767us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074775.587942:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8610 00002000:00100000:3.0:1769074775.587950:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2879488->2883583) req@ffff89b67fa22680 x1855007616874240/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.587957:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22680 with x1855007616874240 ext(2879488->2883583) 00002000:00010000:3.0:1769074775.587961:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa22680 00000100:00100000:1.0:1769074775.589462:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:0.0:1769074775.589868:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074774 -> 1769074775 00002000:00010000:0.0:1769074775.589874:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074774 -> 1769074775 00000100:00100000:0.0:1769074775.589894:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625153920:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074775.589930:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd53e13 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be61 expref: 31 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.589953:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd53e13 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be61 expref: 31 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.589965:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f800, rc 0) 00000100:00100000:0.0:1769074775.590016:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2409:x1855007616873984:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7339us (15245us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074775.590024:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14134 00000100:00100000:0.0:1769074775.590051:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874368 00000100:00100000:0.0:1769074775.590058:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874368 00000100:00100000:0.0:1769074775.590096:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14135 00000100:00100000:0.0:1769074775.590105:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616874368:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074775.590117:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074775.590131:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e1a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be6f expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.590157:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e1a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be6f expref: 29 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.590168:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e1a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be6f expref: 29 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.590176:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:0.0:1769074775.590197:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616874368:12345-192.168.204.60@tcp:101:cat.0 Request processed in 94us (761us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074775.590204:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14135 00000100:00100000:1.0:1769074775.604494:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.604533:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.604589:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.604864:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874496 00000100:00100000:2.0:1769074775.604876:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874496 00000100:00100000:2.0:1769074775.604933:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874624 00000100:00100000:2.0:1769074775.604937:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874624 00000100:00100000:2.0:1769074775.604962:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874752 00000100:00100000:2.0:1769074775.604965:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874752 00000100:00100000:2.0:1769074775.604989:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14136 00000100:00100000:2.0:1769074775.604998:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2409:x1855007616874496:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.605022:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.605050:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53e21 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be76 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.605124:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625154048:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074775.605225:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.605250:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14137 00000100:00100000:2.0:1769074775.605267:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2409:x1855007616874624:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.605276:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.605286:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e28 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be7d expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.605310:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e28 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be7d expref: 33 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.605336:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e28 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be7d expref: 33 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.605351:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ac00, rc 0) 00000100:00100000:2.0:1769074775.605387:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2409:x1855007616874624:12345-192.168.204.60@tcp:101:cat.0 Request processed in 130us (853us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.605394:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14137 00000100:00100000:2.0:1769074775.605414:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14138 00000100:00100000:2.0:1769074775.605421:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616874752:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.605428:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.605434:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e2f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be84 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.605451:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e2f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be84 expref: 29 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.605462:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e2f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be84 expref: 29 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.605470:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ae00, rc 0) 00000100:00100000:2.0:1769074775.605491:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616874752:12345-192.168.204.60@tcp:101:cat.0 Request processed in 70us (902us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.605496:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14138 00000100:00100000:2.0:1769074775.623024:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625154048:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074775.623067:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53e21 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be76 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.623102:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53e21 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be76 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.623114:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773be00, rc 0) 00000100:00100000:2.0:1769074775.623170:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2409:x1855007616874496:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18172us (18687us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.623179:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14136 00000100:00100000:1.0:1769074775.705620:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.705723:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616874880 00000100:00100000:2.0:1769074775.705734:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616874880 00000100:00100000:2.0:1769074775.705942:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14139 00000100:00100000:2.0:1769074775.705954:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+20:62933:x1855007616874880:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074775.705978:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.705998:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be8b expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.706026:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.706035:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 27 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.706041:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.706043:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.706046:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1318912->1855487] (req 1318912->1855487) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.706079:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 27 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.706091:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 27 pid: 8396 timeout: 1863 lvb_type: 1 00000100:00100000:2.0:1769074775.706103:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625154176:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.706196:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.706205:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.706212:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625154304:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.706235:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.712229:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625154176:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.712244:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625154304:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.712288:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1318912->1855487] (req 1318912->1855487) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.712306:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773be00, rc 0) 00000100:00100000:2.0:1769074775.712368:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62933:x1855007616874880:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 6416us (6758us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.712380:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14139 00000100:00100000:0.0:1769074775.714533:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.714648:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875008 00000100:00100000:2.0:1769074775.714658:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875008 00010000:00010000:2.0:1769074775.714703:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 28 pid: 8396 timeout: 1863 lvb_type: 1 00000100:00100000:2.0:1769074775.714785:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7067 00000100:00100000:2.0:1769074775.714795:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616875008:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.714853:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074775.714878:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 28 pid: 8396 timeout: 1863 lvb_type: 1 00010000:00010000:2.0:1769074775.714892:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 28 pid: 8396 timeout: 1863 lvb_type: 1 00010000:00010000:2.0:1769074775.714916:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c400/0x7fe2323adfd53c61 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd5e expref: 28 pid: 8396 timeout: 1863 lvb_type: 1 00010000:00010000:2.0:1769074775.714934:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.714988:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616875008:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 195us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.714996:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7067 00000100:00100000:1.0:1769074775.720280:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.720392:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875136 00000100:00100000:2.0:1769074775.720405:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875136 00010000:00010000:2.0:1769074775.720448:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.720495:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7068 00000100:00100000:2.0:1769074775.720504:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616875136:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.720520:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074775.720540:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.720552:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.720575:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd53c14 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bd2d expref: 21 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.720591:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.720593:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.720601:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.720622:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.720660:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.720674:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625154432:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.720759:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.724029:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625154432:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.724055:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.724118:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2409:x1855007616875136:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3614us (3858us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.724129:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7068 00000100:00100000:1.0:1769074775.725389:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.725482:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875264 00000100:00100000:0.0:1769074775.725490:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875264 00002000:00100000:0.0:1769074775.725537:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1318912->1323007) req@ffff89b67fa23800 x1855007616875264/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.725547:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa23800 with x1855007616875264 ext(1318912->1323007) 00002000:00010000:0.0:1769074775.725552:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fa23800 00000100:00100000:0.0:1769074775.725583:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8611 00000100:00100000:0.0:1769074775.725592:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616875264:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074775.725762:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2408:x1855007616875264:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 171us (381us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074775.725770:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8611 00002000:00100000:0.0:1769074775.725777:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1318912->1323007) req@ffff89b67fa23800 x1855007616875264/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.725786:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa23800 with x1855007616875264 ext(1318912->1323007) 00002000:00010000:0.0:1769074775.725791:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fa23800 00000100:00100000:1.0:1769074775.766312:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.766413:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875392 00000100:00100000:3.0:1769074775.766427:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875392 00002000:00100000:3.0:1769074775.766483:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1851392->1855487) req@ffff89b67fa21880 x1855007616875392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.766495:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa21880 with x1855007616875392 ext(1851392->1855487) 00002000:00010000:3.0:1769074775.766502:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fa21880 00000100:00100000:3.0:1769074775.766537:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8612 00000100:00100000:3.0:1769074775.766547:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616875392:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074775.766906:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616875392:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 359us (607us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074775.766914:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8612 00002000:00100000:3.0:1769074775.766922:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1851392->1855487) req@ffff89b67fa21880 x1855007616875392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/4536 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074775.766930:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa21880 with x1855007616875392 ext(1851392->1855487) 00002000:00010000:3.0:1769074775.766933:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fa21880 00000100:00100000:1.0:1769074775.789655:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.789854:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875648 00000100:00100000:2.0:1769074775.789865:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875648 00000100:00100000:2.0:1769074775.789926:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13330 00000100:00100000:2.0:1769074775.789936:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+241:62942:x1855007616875648:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.789962:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.789983:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e3d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be92 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.790060:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790086:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790097:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790112:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790119:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790124:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790226:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53e52 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790237:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53e52 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790244:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53e52 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074775.790253:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074775.790346:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790352:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790357:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790362:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790369:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773ac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.790373:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790379:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790391:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074775.790395:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.790399:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790405:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ac00/0x7fe2323adfd53e44 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790423:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790428:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790433:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790438:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773b400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074775.790441:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790446:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790452:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4300) 00010000:00010000:2.0:1769074775.790455:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074775.790457:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790462:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b400/0x7fe2323adfd53e4b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074775.790502:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e3d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be92 expref: 244 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.790516:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53e52 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be92 expref: 243 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.790522:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074775.790525:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074775.790527:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773aa00, rc 0) 00000100:00100000:2.0:1769074775.790594:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62942:x1855007616875648:12345-192.168.204.60@tcp:101:cat.0 Request processed in 659us (949us total) trans 8589943967 rc 0/0 00000100:00100000:2.0:1769074775.790848:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13330 00000100:00100000:3.0:1769074775.807575:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.807629:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.807664:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.807695:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875776 00000100:00100000:3.0:1769074775.807704:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875776 00000100:00100000:3.0:1769074775.807754:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616875904 00000100:00100000:3.0:1769074775.807757:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616875904 00000100:00100000:3.0:1769074775.807777:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14140 00000100:00100000:3.0:1769074775.807786:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2408:x1855007616875776:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074775.807843:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074775.807865:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53e60 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879be99 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074775.807920:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625154560:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074775.808003:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074775.808089:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.808125:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.808398:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616876544 00000100:00100000:0.0:1769074775.808404:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616876544 00000100:00100000:0.0:1769074775.808447:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13331 00000100:00100000:0.0:1769074775.808455:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+243:62938:x1855007616876544:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074775.808471:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074775.808489:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd53e67 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bea7 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.808564:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808579:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808591:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808618:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808626:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808633:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808748:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e7c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808759:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e7c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808766:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e7c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074775.808780:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:0.0:1769074775.808911:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808919:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808924:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808931:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808938:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074775.808942:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808950:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808964:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074775.808967:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074775.808972:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808978:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53e6e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808992:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.808997:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.809006:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.809010:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074775.809013:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.809019:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.809025:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b549f47b00) 00010000:00010000:0.0:1769074775.809027:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074775.809029:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.809035:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53e75 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074775.809076:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd53e67 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bea7 expref: 246 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.809088:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e7c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bea7 expref: 245 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.809094:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074775.809096:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769074775.809098:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:0.0:1769074775.809152:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62938:x1855007616876544:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 698us (1522us total) trans 8589943968 rc 0/0 00000100:00100000:0.0:1769074775.809159:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13331 00000100:00100000:2.0:1769074775.809556:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616876288 00000100:00100000:2.0:1769074775.809564:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616876288 00000100:00100000:2.0:1769074775.809632:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1020 00000100:00100000:2.0:1769074775.809642:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:2406:x1855007616876288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.809656:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074775.809677:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773aa00/0x7fe2323adfd53e52 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be92 expref: 245 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.809689:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074775.809692:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.809722:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:2406:x1855007616876288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 83us (1597us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.809729:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1020 00000100:00100000:2.0:1769074775.809754:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14141 00000100:00100000:2.0:1769074775.809762:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2408:x1855007616875904:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.809775:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.809790:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e8a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bea0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.809879:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e8a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bea0 expref: 33 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.809897:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e8a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bea0 expref: 33 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.809905:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ae00, rc 0) 00000100:00100000:2.0:1769074775.809926:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2408:x1855007616875904:12345-192.168.204.60@tcp:101:cat.0 Request processed in 164us (2261us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.809931:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14141 00000100:00100000:1.0:1769074775.810762:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616876160 00000100:00100000:1.0:1769074775.810770:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616876160 00000100:00100000:1.0:1769074775.810840:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4678 00000100:00100000:1.0:1769074775.810849:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:62942:x1855007616876160:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074775.811050:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62942:x1855007616876160:12345-192.168.204.60@tcp:35:cat.0 Request processed in 202us (2960us total) trans 8589943969 rc 0/0 00000100:00100000:1.0:1769074775.811056:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4678 00000100:00100000:0.0:1769074775.818652:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625154560:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074775.818696:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53e60 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be99 expref: 31 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.818721:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53e60 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be99 expref: 31 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074775.818738:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4a00, rc 0) 00000100:00100000:0.0:1769074775.818886:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2408:x1855007616875776:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11099us (11319us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074775.818895:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14140 00000100:00100000:3.0:1769074775.823855:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074775.823894:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074775.823995:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616876672 00000100:00100000:1.0:1769074775.824007:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616876672 00000100:00100000:1.0:1769074775.824065:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616876800 00000100:00100000:1.0:1769074775.824068:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616876800 00000100:00100000:1.0:1769074775.824111:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14142 00000100:00100000:1.0:1769074775.824120:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616876672:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074775.824137:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074775.824156:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd53e91 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879beae expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074775.824223:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625154688:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074775.824321:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abbe40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.824405:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14143 00000100:00100000:2.0:1769074775.824416:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616876800:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.824424:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.824436:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e98 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879beb5 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.824459:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e98 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beb5 expref: 29 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.824477:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e98 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beb5 expref: 29 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074775.824489:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ae00, rc 0) 00000100:00100000:2.0:1769074775.824520:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2407:x1855007616876800:12345-192.168.204.60@tcp:101:cat.0 Request processed in 106us (625us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074775.824527:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14143 00000100:00100000:1.0:1769074775.828220:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.828336:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616876928 00000100:00100000:0.0:1769074775.828348:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616876928 00000100:00100000:0.0:1769074775.828415:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1021 00000100:00100000:0.0:1769074775.828425:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:2408:x1855007616876928:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074775.828447:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074775.828474:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f400/0x7fe2323adfd53e7c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bea7 expref: 245 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.828491:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074775.828495:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074775.828554:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:2408:x1855007616876928:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 131us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.828561:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1021 00000100:00100000:3.0:1769074775.837266:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.837444:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877184 00000100:00100000:2.0:1769074775.837456:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877184 00000100:00100000:2.0:1769074775.837518:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4679 00000100:00100000:2.0:1769074775.837527:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c6450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+244:62938:x1855007616877184:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074775.837765:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c6450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62938:x1855007616877184:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 238us (522us total) trans 8589943970 rc 0/0 00000100:00100000:2.0:1769074775.837773:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4679 00002000:00010000:1.0:1769074775.838064:0:36516:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074773 -> 1769074775 00002000:00010000:1.0:1769074775.838069:0:36516:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074773 -> 1769074775 00002000:00010000:1.0:1769074775.838071:0:36516:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074773 -> 1769074775 00000100:00100000:1.0:1769074775.838087:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625154688:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074775.838120:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd53e91 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beae expref: 28 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074775.838140:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd53e91 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beae expref: 28 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074775.838150:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44000, rc 0) 00000100:00100000:1.0:1769074775.838175:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616876672:12345-192.168.204.60@tcp:101:cat.0 Request processed in 14057us (14375us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074775.838183:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14142 00000100:00100000:3.0:1769074775.843839:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.843941:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877312 00000100:00100000:0.0:1769074775.843950:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877312 00002000:00100000:0.0:1769074775.843999:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5444a5f80 x1855007616877312/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 504/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.844008:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5f80 with x1855007616877312 ext(1048576->2097151) 00002000:00010000:0.0:1769074775.844014:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5444a5f80 00000100:00100000:0.0:1769074775.844050:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8613 00000100:00100000:0.0:1769074775.844059:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616877312:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074775.862773:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.862962:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877440 00000100:00100000:2.0:1769074775.862974:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877440 00000100:00100000:2.0:1769074775.863037:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14144 00000100:00100000:2.0:1769074775.863047:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:62942:x1855007616877440:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.863067:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.863101:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bebc expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.863134:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.863143:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.863145:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.863149:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.863186:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 21 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.863203:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 21 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.863223:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625154816:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.863319:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074775.863605:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616877312:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19546us (19805us total) trans 0 rc 512000/512000 00000100:00100000:0.0:1769074775.863617:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8613 00002000:00100000:0.0:1769074775.863631:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5444a5f80 x1855007616877312/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 504/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 512000/512000 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.863640:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5444a5f80 with x1855007616877312 ext(1048576->2097151) 00010000:00010000:0.0:1769074775.863647:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.863660:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:0.0:1769074775.863667:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5444a5f80 00000100:00100000:2.0:1769074775.868347:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625154816:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.868401:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 28 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.868431:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ae00, rc 0) 00000100:00100000:2.0:1769074775.868521:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62942:x1855007616877440:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5471us (5756us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.868532:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14144 00000100:00100000:1.0:1769074775.876314:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.876535:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877568 00000100:00100000:0.0:1769074775.876546:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877568 00002000:00100000:0.0:1769074775.876596:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1318912->1855487) req@ffff89b67fa23100 x1855007616877568/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.876620:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa23100 with x1855007616877568 ext(1318912->1855487) 00010000:00010000:0.0:1769074775.876630:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.876645:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 20 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:0.0:1769074775.876652:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fa23100 00000100:00100000:0.0:1769074775.876688:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 794 00000100:00100000:0.0:1769074775.876699:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+20:2407:x1855007616877568:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074775.878837:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.878934:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877696 00000100:00100000:2.0:1769074775.878947:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877696 00000100:00100000:2.0:1769074775.879003:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14145 00000100:00100000:2.0:1769074775.879013:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62944:x1855007616877696:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074775.879029:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074775.879048:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bec3 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.879091:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.879099:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.879102:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074775.879106:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 31 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.879148:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 27 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.879162:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 27 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.879174:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625154944:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074775.879261:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074775.888997:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625154944:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074775.889046:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 31 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.889071:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773aa00, rc 0) 00000100:00100000:2.0:1769074775.889165:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62944:x1855007616877696:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10150us (10361us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.889176:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14145 00000100:00100000:1.0:1769074775.897691:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.899277:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877824 00000100:00100000:2.0:1769074775.899299:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877824 00002000:00100000:2.0:1769074775.899376:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1970176->1978367) req@ffff89b5436d4e00 x1855007616877824/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 8680/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.899392:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d4e00 with x1855007616877824 ext(1970176->1978367) 00010000:00010000:2.0:1769074775.899405:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 28 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.899429:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 28 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074775.899439:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5436d4e00 00000100:00100000:2.0:1769074775.899476:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 795 00000100:00100000:2.0:1769074775.899486:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616877824:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074775.900977:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2407:x1855007616877568:12345-192.168.204.60@tcp:4:cat.0 Request processed in 24270us (24667us total) trans 21474841203 rc 0/0 00000100:00100000:0.0:1769074775.900989:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 794 00002000:00100000:0.0:1769074775.901000:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bb:0x0] (1318912->1855487) req@ffff89b67fa23100 x1855007616877568/t21474841203(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/448 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.901009:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa23100 with x1855007616877568 ext(1318912->1855487) 00010000:00010000:0.0:1769074775.901019:0:28448:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 22 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.901033:0:28448:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 22 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:0.0:1769074775.901040:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fa23100 00000100:00100000:2.0:1769074775.902031:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616877824:12345-192.168.204.60@tcp:4:cat.0 Request processed in 2537us (4350us total) trans 17179874522 rc 0/0 00000100:00100000:2.0:1769074775.902043:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 795 00002000:00100000:2.0:1769074775.902055:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1970176->1978367) req@ffff89b5436d4e00 x1855007616877824/t17179874522(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 8680/448 e 0 to 0 dl 1769074786 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.902065:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d4e00 with x1855007616877824 ext(1970176->1978367) 00010000:00010000:2.0:1769074775.902097:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.902112:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074775.902120:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5436d4e00 00000100:00100000:1.0:1769074775.907928:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.908459:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616877952 00000100:00100000:2.0:1769074775.908474:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616877952 00010000:00010000:2.0:1769074775.908520:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074775.908568:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7069 00000100:00100000:2.0:1769074775.908578:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616877952:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074775.908596:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074775.908635:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.908647:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.908671:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b640031200/0x7fe2323adfd53daa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1970176->1978367) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be1b expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074775.908691:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.908693:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074775.908701:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.908711:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074775.908757:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074775.908774:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625155072:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074775.909150:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074775.914311:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.914389:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878080 00000100:00100000:0.0:1769074775.914400:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878080 00010000:00010000:0.0:1769074775.914439:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 22 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:0.0:1769074775.914490:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7070 00000100:00100000:0.0:1769074775.914500:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+22:2406:x1855007616878080:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074775.914516:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074775.914538:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 22 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.914551:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 22 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.914576:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773be00/0x7fe2323adfd53e36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1318912->1855487) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be8b expref: 22 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:0.0:1769074775.914591:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074775.914594:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074775.914613:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.914623:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074775.914663:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 27 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074775.914678:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625155200:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074775.914767:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59240 going to sleep for 11 seconds 00000100:00100000:0.0:1769074775.920493:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625155200:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074775.920524:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074775.920585:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616878080:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6086us (6285us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074775.920596:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7070 00000100:00100000:2.0:1769074775.920871:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625155072:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074775.920894:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074775.920928:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616877952:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12350us (13011us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074775.920937:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7069 00000100:00100000:3.0:1769074775.924345:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074775.924475:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878208 00000100:00100000:2.0:1769074775.924487:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878208 00002000:00100000:2.0:1769074775.924556:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54c844380 x1855007616878208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.924569:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c844380 with x1855007616878208 ext(1048576->2097151) 00002000:00010000:2.0:1769074775.924575:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c844380 00000100:00100000:2.0:1769074775.924616:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8614 00000100:00100000:2.0:1769074775.924626:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616878208:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074775.942201:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074775.944899:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878336 00000100:00100000:0.0:1769074775.944917:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878336 00002000:00100000:0.0:1769074775.944989:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5436d6d80 x1855007616878336/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/0 e 0 to 0 dl 1769074786 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.945002:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6d80 with x1855007616878336 ext(1048576->2097151) 00002000:00010000:0.0:1769074775.945008:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6d80 00000100:00100000:0.0:1769074775.945037:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8615 00000100:00100000:0.0:1769074775.945048:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2406:x1855007616878336:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074775.958737:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616878208:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34097us (34389us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074775.958753:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8614 00002000:00100000:2.0:1769074775.958767:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54c844380 x1855007616878208/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074775.958777:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c844380 with x1855007616878208 ext(1048576->2097151) 00002000:00010000:2.0:1769074775.958784:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54c844380 00000100:00100000:0.0:1769074775.966414:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2406:x1855007616878336:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21348us (24204us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074775.966431:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8615 00002000:00100000:0.0:1769074775.966460:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5436d6d80 x1855007616878336/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:311/0 lens 488/440 e 0 to 0 dl 1769074786 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074775.966472:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6d80 with x1855007616878336 ext(1048576->2097151) 00002000:00010000:0.0:1769074775.966480:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6d80 00000100:00100000:1.0:1769074776.001263:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.001307:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.001399:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878464 00000100:00100000:2.0:1769074776.001408:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878464 00000100:00100000:2.0:1769074776.001470:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878592 00000100:00100000:2.0:1769074776.001474:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878592 00000100:00100000:2.0:1769074776.001506:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14146 00000100:00100000:2.0:1769074776.001515:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62942:x1855007616878464:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.001534:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.001552:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879beca expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001577:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.001579:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.001588:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001614:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001627:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b400, rc 0) 00000100:00100000:2.0:1769074776.001679:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:62942:x1855007616878464:12345-192.168.204.60@tcp:101:cat.0 Request processed in 165us (440us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.001686:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14146 00000100:00100000:2.0:1769074776.001707:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14147 00000100:00100000:2.0:1769074776.001714:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62944:x1855007616878592:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.001721:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.001731:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bed1 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001751:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001760:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.001763:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.001769:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 32 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001854:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 29 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.001870:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 29 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074776.001886:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625155328:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.001953:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.009238:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625155328:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074776.009304:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 32 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.009323:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ac00, rc 0) 00000100:00100000:2.0:1769074776.009389:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62944:x1855007616878592:12345-192.168.204.60@tcp:101:cat.0 Request processed in 7675us (8081us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.009399:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14147 00000100:00100000:3.0:1769074776.023551:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.023647:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.023967:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878720 00000100:00100000:2.0:1769074776.023981:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878720 00002000:00100000:2.0:1769074776.024044:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2392064->2883583) req@ffff89b54c845f80 x1855007616878720/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.024057:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845f80 with x1855007616878720 ext(2392064->2883583) 00010000:00010000:2.0:1769074776.024202:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.024232:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 30 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074776.024242:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54c845f80 00000100:00100000:2.0:1769074776.024298:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878848 00000100:00100000:2.0:1769074776.024306:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878848 00002000:00100000:2.0:1769074776.024335:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54441d500 x1855007616878848/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.024345:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441d500 with x1855007616878848 ext(1048576->2097151) 00002000:00010000:2.0:1769074776.024350:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441d500 00000100:00100000:2.0:1769074776.024400:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 796 00000100:00100000:2.0:1769074776.024414:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616878720:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074776.024886:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8616 00000100:00100000:1.0:1769074776.024901:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616878848:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074776.043993:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2409:x1855007616878720:12345-192.168.204.60@tcp:4:cat.0 Request processed in 19572us (20443us total) trans 17179874523 rc 0/0 00000100:00100000:2.0:1769074776.044008:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 796 00002000:00100000:2.0:1769074776.044021:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2392064->2883583) req@ffff89b54c845f80 x1855007616878720/t17179874523(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/448 e 0 to 0 dl 1769074787 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.044031:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c845f80 with x1855007616878720 ext(2392064->2883583) 00010000:00010000:2.0:1769074776.044041:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 34 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.044059:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 34 pid: 27843 timeout: 1863 lvb_type: 0 00002000:00010000:2.0:1769074776.044718:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54c845f80 00000100:00100000:1.0:1769074776.065288:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.065570:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616878848:12345-192.168.204.60@tcp:3:cat.0 Request processed in 40672us (41922us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074776.065585:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8616 00002000:00100000:0.0:1769074776.065610:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54441d500 x1855007616878848/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.065621:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54441d500 with x1855007616878848 ext(1048576->2097151) 00002000:00010000:0.0:1769074776.065627:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54441d500 00000100:00100000:2.0:1769074776.065859:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616878976 00000100:00100000:2.0:1769074776.065868:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616878976 00010000:00010000:2.0:1769074776.065903:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 32 pid: 27843 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074776.065940:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7071 00000100:00100000:2.0:1769074776.065950:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616878976:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074776.065965:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074776.065983:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 32 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.065997:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 32 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.066032:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53e05 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2392064->2883583] (req 2392064->2883583) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be5a expref: 32 pid: 27843 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.066056:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.066060:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.066090:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.066102:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.066161:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.066181:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625155456:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074776.066276:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.085014:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625155456:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074776.085043:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074776.085120:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616878976:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 19169us (19844us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.085133:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7071 00000100:00100000:3.0:1769074776.090030:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.090206:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879104 00000100:00100000:2.0:1769074776.090218:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879104 00000100:00100000:2.0:1769074776.090285:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14148 00000100:00100000:2.0:1769074776.090295:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:62942:x1855007616879104:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.090317:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.090339:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bed8 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.090371:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.090375:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.090385:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 32 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.090403:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 32 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.090418:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ba00, rc 0) 00000100:00100000:2.0:1769074776.090510:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62942:x1855007616879104:12345-192.168.204.60@tcp:101:cat.0 Request processed in 214us (490us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.090518:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14148 00000100:00100000:1.0:1769074776.103626:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.103882:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879232 00000100:00100000:0.0:1769074776.103897:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879232 00002000:00100000:0.0:1769074776.103961:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5436d4000 x1855007616879232/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.103973:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d4000 with x1855007616879232 ext(2097152->4194303) 00002000:00010000:0.0:1769074776.103981:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d4000 00000100:00100000:0.0:1769074776.104027:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8617 00000100:00100000:0.0:1769074776.104041:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2407:x1855007616879232:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074776.129642:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.130217:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879360 00000100:00100000:2.0:1769074776.130231:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879360 00002000:00100000:2.0:1769074776.130309:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b6656d9180 x1855007616879360/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.130321:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d9180 with x1855007616879360 ext(2097152->4194303) 00002000:00010000:2.0:1769074776.130328:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6656d9180 00000100:00100000:2.0:1769074776.130362:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8618 00000100:00100000:2.0:1769074776.130381:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616879360:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074776.142560:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.142762:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879488 00000100:00100000:3.0:1769074776.142773:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879488 00000100:00080000:1.0:1769074776.143003:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074776.143703:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2407:x1855007616879232:12345-192.168.204.60@tcp:3:cat.0 Request processed in 39661us (40086us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074776.143716:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8617 00002000:00100000:2.0:1769074776.143730:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b5436d4000 x1855007616879232/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.143739:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d4000 with x1855007616879232 ext(2097152->4194303) 00002000:00010000:2.0:1769074776.143745:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d4000 00000100:00100000:0.0:1769074776.150236:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14149 00000100:00100000:0.0:1769074776.150271:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:62938:x1855007616879488:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074776.150299:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074776.150331:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bedf expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.150372:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.150386:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074776.150389:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074776.150393:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1441792->2097151] (req 1441792->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.150444:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 29 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.150459:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 29 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:0.0:1769074776.150476:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625155584:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074776.150613:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59240 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.153442:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616879360:12345-192.168.204.60@tcp:3:cat.0 Request processed in 23056us (23797us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074776.153458:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8618 00002000:00100000:2.0:1769074776.153471:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b6656d9180 x1855007616879360/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.153483:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d9180 with x1855007616879360 ext(2097152->4194303) 00002000:00010000:2.0:1769074776.153489:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6656d9180 00000100:00100000:3.0:1769074776.162611:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.162681:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.162742:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879744 00000100:00100000:1.0:1769074776.162756:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879744 00000100:00100000:1.0:1769074776.162862:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14150 00000100:00100000:1.0:1769074776.162872:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616879744:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074776.162896:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074776.162920:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53ec9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bee6 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.162946:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.162961:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53ec9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bee6 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.162985:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd53ec9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bee6 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.162998:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:1.0:1769074776.163063:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616879744:12345-192.168.204.60@tcp:101:cat.0 Request processed in 192us (381us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074776.163070:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14150 00000100:00100000:2.0:1769074776.163076:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879616 00000100:00100000:2.0:1769074776.163081:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879616 00010000:00010000:2.0:1769074776.163103:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:2.0:1769074776.163126:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7072 00000100:00100000:2.0:1769074776.163133:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616879616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074776.163143:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074776.163165:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.163177:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00010000:00010000:2.0:1769074776.163197:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.163199:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.163209:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 30 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.163219:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 30 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.163269:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 30 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.163283:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625155712:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074776.163362:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00010000:00010000:0.0:1769074776.168404:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773ae00/0x7fe2323adfd53e9f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bebc expref: 30 pid: 36516 timeout: 1863 lvb_type: 0 00000100:00100000:0.0:1769074776.168426:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625155584:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074776.168472:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 29 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.168489:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b7200, rc 0) 00000100:00100000:0.0:1769074776.168603:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:62938:x1855007616879488:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 18344us (26050us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.168612:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14149 00000100:00100000:1.0:1769074776.169981:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.170063:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625155712:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074776.170093:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616879872 00010000:00010000:0.0:1769074776.170097:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074776.170100:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616879872 00000100:00100000:0.0:1769074776.170152:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616879616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7018us (7566us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.170163:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7072 00000100:00100000:2.0:1769074776.170354:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14151 00000100:00100000:2.0:1769074776.170365:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616879872:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.170388:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.170410:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53ed0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879beed expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.170492:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625155840:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074776.170574:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074776.174052:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.174475:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880000 00000100:00100000:2.0:1769074776.174487:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880000 00002000:00100000:2.0:1769074776.174553:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1441792->1445887) req@ffff89b5436d7800 x1855007616880000/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.174578:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7800 with x1855007616880000 ext(1441792->1445887) 00002000:00010000:2.0:1769074776.174586:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d7800 00000100:00100000:0.0:1769074776.174936:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625155840:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074776.174978:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53ed0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beed expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074776.174999:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773b200/0x7fe2323adfd53ed0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beed expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074776.175011:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b200, rc 0) 00000100:00100000:0.0:1769074776.175083:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616879872:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4703us (5098us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074776.175093:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14151 00000100:00100000:2.0:1769074776.175350:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8619 00000100:00100000:2.0:1769074776.175362:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616880000:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074776.175545:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616880000:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 185us (1504us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074776.176662:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8619 00002000:00100000:2.0:1769074776.176678:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1441792->1445887) req@ffff89b5436d7800 x1855007616880000/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/4536 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.176689:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7800 with x1855007616880000 ext(1441792->1445887) 00002000:00010000:2.0:1769074776.176696:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d7800 00000100:00100000:1.0:1769074776.189688:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.189877:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880128 00000100:00100000:0.0:1769074776.189890:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880128 00000100:00100000:0.0:1769074776.189955:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14152 00000100:00100000:0.0:1769074776.189967:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616880128:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074776.189988:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074776.190014:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd53ed7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bef4 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074776.190096:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625155968:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074776.190199:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.201070:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625155968:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074776.201374:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd53ed7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bef4 expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074776.201401:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6200/0x7fe2323adfd53ed7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bef4 expref: 22 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074776.201429:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5481b6200, rc 0) 00000100:00100000:2.0:1769074776.201514:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2409:x1855007616880128:12345-192.168.204.60@tcp:101:cat.0 Request processed in 11549us (11838us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074776.201525:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14152 00000100:00100000:1.0:1769074776.213222:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.213339:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880256 00000100:00100000:2.0:1769074776.213351:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880256 00000100:00100000:2.0:1769074776.213422:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14153 00000100:00100000:2.0:1769074776.213432:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62938:x1855007616880256:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074776.213451:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.213472:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879befb expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.213514:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 31 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.213522:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.213529:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.213530:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.213534:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->1273855] (req 1048576->1273855) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 33 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.213567:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.213579:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.213593:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625156096:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074776.213701:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 31 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.213710:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 31 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.213718:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625156224:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.213740:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.218052:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625156096:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.218069:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625156224:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074776.218103:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->1273855] (req 1048576->1273855) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 33 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.218123:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773a000, rc 0) 00000100:00100000:2.0:1769074776.218201:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:62938:x1855007616880256:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 4770us (4990us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.218214:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14153 00000100:00100000:0.0:1769074776.228218:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.228340:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880384 00000100:00100000:2.0:1769074776.228351:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880384 00010000:00010000:2.0:1769074776.229038:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.229093:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7073 00000100:00100000:2.0:1769074776.229106:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2408:x1855007616880384:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074776.229127:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074776.229151:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.229166:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.229193:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b400/0x7fe2323adfd53ead lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879beca expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.229212:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074776.229400:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616880384:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 289us (1186us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.229415:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7073 00000100:00100000:1.0:1769074776.235205:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.235386:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880512 00000100:00100000:0.0:1769074776.235416:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880512 00010000:00010000:0.0:1769074776.235475:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 32 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:0.0:1769074776.235553:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7074 00000100:00100000:0.0:1769074776.235565:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2407:x1855007616880512:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074776.235584:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074776.235610:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 32 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.235626:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 32 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.235653:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773aa00/0x7fe2323adfd53ea6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bec3 expref: 32 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.235669:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074776.235672:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074776.235679:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.235688:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.235726:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074776.235745:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625156352:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074776.236147:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.239026:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625156352:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074776.239052:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074776.239111:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2407:x1855007616880512:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3548us (3915us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.239122:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7074 00000100:00100000:2.0:1769074776.244486:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.244625:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880640 00000100:00100000:0.0:1769074776.244640:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880640 00002000:00100000:0.0:1769074776.244695:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1269760->1273855) req@ffff89b67b413480 x1855007616880640/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.244707:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b413480 with x1855007616880640 ext(1269760->1273855) 00002000:00010000:0.0:1769074776.244713:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67b413480 00000100:00100000:0.0:1769074776.244749:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8620 00000100:00100000:0.0:1769074776.244758:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616880640:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074776.244992:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616880640:12345-192.168.204.60@tcp:3:cat.0 Request processed in 234us (517us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074776.245001:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8620 00002000:00100000:0.0:1769074776.245008:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1269760->1273855) req@ffff89b67b413480 x1855007616880640/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/4536 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.245017:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b413480 with x1855007616880640 ext(1269760->1273855) 00002000:00010000:0.0:1769074776.245020:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67b413480 00000100:00100000:1.0:1769074776.347007:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.347541:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616880896 00000100:00100000:3.0:1769074776.347553:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616880896 00000100:00100000:3.0:1769074776.347644:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13332 00000100:00100000:3.0:1769074776.347654:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+245:62953:x1855007616880896:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.347683:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.347710:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53ee5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf02 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.347882:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.347901:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.347916:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.347933:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.347942:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.347949:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348486:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53efa lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348498:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53efa lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348505:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53efa lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:3.0:1769074776.348515:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074776.348708:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348718:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348724:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348731:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348738:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074776.348742:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348749:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348763:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074776.348767:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074776.348772:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348778:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53eec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348792:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.348798:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00000100:00080000:1.0:1769074776.348948:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1769074776.349090:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.349099:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea5400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074776.349102:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.349109:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.349118:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6439ad800) 00010000:00010000:3.0:1769074776.349121:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074776.349209:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.349218:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5400/0x7fe2323adfd53ef3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074776.349309:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53ee5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf02 expref: 248 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.349323:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53efa lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf02 expref: 247 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.349331:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074776.349334:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074776.349336:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4000, rc 0) 00000100:00100000:3.0:1769074776.349417:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62953:x1855007616880896:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1763us (2421us total) trans 8589943971 rc 0/0 00000100:00100000:3.0:1769074776.349425:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13332 00000100:00100000:1.0:1769074776.358583:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.358679:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.358970:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616881152 00000100:00100000:3.0:1769074776.358981:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616881152 00000100:00100000:3.0:1769074776.359047:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4680 00000100:00100000:3.0:1769074776.359059:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62953:x1855007616881152:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074776.359308:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62953:x1855007616881152:12345-192.168.204.60@tcp:35:cat.0 Request processed in 252us (733us total) trans 8589943972 rc 0/0 00000100:00100000:3.0:1769074776.359316:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4680 00000100:00100000:0.0:1769074776.359733:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616881280 00000100:00100000:0.0:1769074776.359740:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616881280 00000100:00100000:0.0:1769074776.359785:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1022 00000100:00100000:0.0:1769074776.359793:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:2409:x1855007616881280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074776.359835:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074776.359859:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53efa lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf02 expref: 248 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.359876:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074776.359881:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074776.359911:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:2409:x1855007616881280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 120us (1232us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.359918:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1022 00000100:00100000:1.0:1769074776.382283:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.382919:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616881408 00000100:00100000:2.0:1769074776.382931:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616881408 00000100:00100000:2.0:1769074776.383010:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14154 00000100:00100000:2.0:1769074776.383021:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62947:x1855007616881408:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074776.383039:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.383060:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf09 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.383364:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.383374:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 27 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074776.383381:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.383383:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.383388:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [1081344->2088959] (req 1081344->2088959) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 22 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.383425:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 27 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074776.383437:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 27 pid: 9048 timeout: 1864 lvb_type: 1 00000100:00100000:2.0:1769074776.383451:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625156480:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074776.383554:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 22 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.383563:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 22 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.383573:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625156608:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.384306:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074776.405793:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.405872:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.405925:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.405945:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.405970:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616881792 00000100:00100000:3.0:1769074776.405978:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616881792 00010000:00010000:3.0:1769074776.406014:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 23 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074776.406046:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7075 00000100:00100000:3.0:1769074776.406055:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616881792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074776.406079:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074776.406100:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 23 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:3.0:1769074776.406110:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 23 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:3.0:1769074776.406145:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074776.406189:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+23:2408:x1855007616881792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 135us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.406196:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7075 00000100:00100000:3.0:1769074776.406255:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625156480:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074776.406270:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd53b8f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bce0 expref: 22 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074776.406283:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625156608:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074776.406309:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1081344->2088959] (req 1081344->2088959) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 21 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.406326:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773b800, rc 0) 00000100:00100000:3.0:1769074776.406352:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62947:x1855007616881408:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 23334us (24081us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.406359:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14154 00000100:00100000:0.0:1769074776.406525:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616881536 00000100:00100000:0.0:1769074776.406531:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616881536 00002000:00100000:0.0:1769074776.406564:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1441791) req@ffff89b6656d8a80 x1855007616881536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.406574:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d8a80 with x1855007616881536 ext(1048576->1441791) 00002000:00010000:0.0:1769074776.406578:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6656d8a80 00000100:00100000:0.0:1769074776.406599:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616881664 00000100:00100000:0.0:1769074776.406602:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616881664 00002000:00100000:0.0:1769074776.406617:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1273856->2097151) req@ffff89b6656da300 x1855007616881664/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.406626:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656da300 with x1855007616881664 ext(1273856->2097151) 00002000:00010000:0.0:1769074776.406630:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6656da300 00000100:00100000:0.0:1769074776.406646:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8621 00000100:00100000:0.0:1769074776.406653:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616881536:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074776.406951:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8622 00000100:00100000:0.0:1769074776.406959:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616881664:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074776.408300:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882048 00000100:00100000:2.0:1769074776.408312:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882048 00000100:00100000:2.0:1769074776.408366:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13333 00000100:00100000:2.0:1769074776.408373:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+247:62955:x1855007616882048:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074776.408393:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.408414:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f0f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf10 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.408481:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408495:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408505:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408518:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408525:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408531:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408632:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd53f24 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408642:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd53f24 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408649:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd53f24 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074776.408659:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074776.408733:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408740:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408746:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408752:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408758:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773a800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074776.408762:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408769:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408779:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074776.408783:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074776.408787:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408793:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a800/0x7fe2323adfd53f16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408843:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408849:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408855:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408859:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773a600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074776.408862:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408867:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408873:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4300) 00010000:00010000:2.0:1769074776.408876:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074776.408878:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408884:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd53f1d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.408918:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f0f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf10 expref: 250 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.408930:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd53f24 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf10 expref: 249 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.408937:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074776.408940:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074776.408943:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773a400, rc 0) 00000100:00100000:2.0:1769074776.408994:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62955:x1855007616882048:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 621us (3210us total) trans 8589943973 rc 0/0 00000100:00100000:2.0:1769074776.409001:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13333 00000100:00100000:1.0:1769074776.420111:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.420322:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882176 00000100:00100000:0.0:1769074776.420332:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882176 00010000:00010000:0.0:1769074776.420377:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 30 pid: 9048 timeout: 1864 lvb_type: 1 00000100:00100000:0.0:1769074776.420423:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7076 00000100:00100000:0.0:1769074776.420434:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616882176:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074776.420460:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074776.420482:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 30 pid: 9048 timeout: 1864 lvb_type: 1 00010000:00010000:0.0:1769074776.420496:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 30 pid: 9048 timeout: 1864 lvb_type: 1 00010000:00010000:0.0:1769074776.420519:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910ca00/0x7fe2323adfd53bb9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcee expref: 30 pid: 9048 timeout: 1864 lvb_type: 1 00010000:00010000:0.0:1769074776.420536:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074776.420538:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074776.420546:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.420557:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.420598:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 20 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074776.420611:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625156736:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074776.420697:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074776.420749:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616881536:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14094us (14874us total) trans 0 rc 393216/393216 00000100:00100000:3.0:1769074776.420760:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8621 00002000:00100000:3.0:1769074776.420770:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->1441791) req@ffff89b6656d8a80 x1855007616881536/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 393216/393216 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.420780:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d8a80 with x1855007616881536 ext(1048576->1441791) 00002000:00010000:3.0:1769074776.420785:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b6656d8a80 00000100:00100000:2.0:1769074776.424295:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616881664:12345-192.168.204.60@tcp:3:cat.0 Request processed in 17325us (18357us total) trans 0 rc 823296/823296 00000100:00100000:2.0:1769074776.424310:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8622 00002000:00100000:2.0:1769074776.424322:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1273856->2097151) req@ffff89b6656da300 x1855007616881664/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 823296/823296 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.424333:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656da300 with x1855007616881664 ext(1273856->2097151) 00002000:00010000:2.0:1769074776.424339:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6656da300 00000100:00100000:1.0:1769074776.424482:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625156736:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074776.424507:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074776.424533:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616882176:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4101us (4432us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074776.424541:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7076 00000100:00100000:2.0:1769074776.425317:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.425384:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882304 00000100:00100000:0.0:1769074776.425392:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882304 00002000:00100000:0.0:1769074776.425432:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1081344->1085439) req@ffff89b67b411500 x1855007616882304/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.425441:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b411500 with x1855007616882304 ext(1081344->1085439) 00002000:00010000:0.0:1769074776.425446:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b411500 00000100:00100000:0.0:1769074776.425475:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8623 00000100:00100000:0.0:1769074776.425484:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616882304:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074776.425625:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616882304:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 142us (311us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074776.425633:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8623 00002000:00100000:0.0:1769074776.425639:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1081344->1085439) req@ffff89b67b411500 x1855007616882304/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/4536 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.425646:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b411500 with x1855007616882304 ext(1081344->1085439) 00002000:00010000:0.0:1769074776.425649:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b411500 00000100:00100000:0.0:1769074776.447485:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.447544:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.447628:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882432 00000100:00100000:3.0:1769074776.447644:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882432 00000100:00100000:0.0:1769074776.447653:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.447716:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14155 00000100:00100000:3.0:1769074776.447727:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616882432:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.447749:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.447769:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53f32 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf17 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074776.447783:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882816 00000100:00100000:1.0:1769074776.447787:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882816 00000100:00100000:1.0:1769074776.447863:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1023 00000100:00100000:1.0:1769074776.447871:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:2409:x1855007616882816:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1769074776.447876:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625156864:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074776.447882:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074776.447907:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd53f24 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf10 expref: 249 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.447925:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074776.447931:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074776.447957:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:2409:x1855007616882816:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 87us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.447959:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074776.447965:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1023 00000100:00100000:2.0:1769074776.448040:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882688 00000100:00100000:2.0:1769074776.448044:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882688 00000100:00100000:2.0:1769074776.448080:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4681 00000100:00100000:2.0:1769074776.448087:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+248:62955:x1855007616882688:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:2.0:1769074776.448301:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62955:x1855007616882688:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 214us (756us total) trans 8589943974 rc 0/0 00000100:00100000:2.0:1769074776.448308:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4681 00000100:00100000:2.0:1769074776.461028:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.461125:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616882944 00000100:00100000:0.0:1769074776.461136:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616882944 00002000:00100000:0.0:1769074776.461200:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (2084864->2088959) req@ffff89b67b410380 x1855007616882944/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00010000:3.0:1769074776.461203:0:9048:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime: 1769074772 -> 1769074776 00002000:00010000:3.0:1769074776.461207:0:9048:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074772 -> 1769074775 00002000:00010000:3.0:1769074776.461209:0:9048:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime: 1769074772 -> 1769074776 00002000:00100000:0.0:1769074776.461212:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b410380 with x1855007616882944 ext(2084864->2088959) 00002000:00010000:0.0:1769074776.461217:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b410380 00000100:00100000:3.0:1769074776.461230:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625156864:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074776.461272:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53f32 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf17 expref: 28 pid: 9048 timeout: 0 lvb_type: 1 00000100:00100000:0.0:1769074776.461284:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8624 00000100:00100000:0.0:1769074776.461293:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616882944:12345-192.168.204.60@tcp:3:multiop.0 00010000:00010000:3.0:1769074776.461293:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67fea4e00/0x7fe2323adfd53f32 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf17 expref: 28 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.461305:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea4e00, rc 0) 00000100:00100000:3.0:1769074776.461364:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616882432:12345-192.168.204.60@tcp:101:cat.0 Request processed in 13640us (13896us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074776.461372:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14155 00000100:00100000:0.0:1769074776.461431:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2407:x1855007616882944:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 139us (414us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074776.461437:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8624 00002000:00100000:0.0:1769074776.461443:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (2084864->2088959) req@ffff89b67b410380 x1855007616882944/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/4536 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.461450:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b410380 with x1855007616882944 ext(2084864->2088959) 00002000:00010000:0.0:1769074776.461454:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b410380 00000100:00100000:2.0:1769074776.473718:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.473785:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883072 00000100:00100000:3.0:1769074776.473798:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883072 00000100:00100000:3.0:1769074776.473887:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14156 00000100:00100000:3.0:1769074776.473896:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:62955:x1855007616883072:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074776.473915:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.473935:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf1e expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.473965:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.473975:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.473982:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074776.473985:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074776.473989:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->2154495] (req 2097152->2154495) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 33 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.474021:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.474033:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074776.474046:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625156992:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074776.474145:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.474154:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 30 pid: 8398 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074776.474164:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625157120:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074776.474189:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00010000:00010000:3.0:1769074776.477030:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074776.488262:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.488515:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625156992:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074776.488597:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883200 00000100:00100000:0.0:1769074776.488605:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883200 00000100:00100000:0.0:1769074776.488677:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14157 00000100:00100000:0.0:1769074776.488688:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616883200:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074776.488710:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074776.488730:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53f40 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf25 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074776.488788:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625157248:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074776.488901:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074776.497425:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625157120:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074776.497468:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->2154495] (req 2097152->2154495) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 33 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.497488:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea5400, rc 0) 00000100:00100000:3.0:1769074776.497551:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:62955:x1855007616883072:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 23655us (23842us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.497561:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14156 00000100:00100000:0.0:1769074776.497690:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625157248:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074776.497719:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53f40 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf25 expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074776.497735:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e47c00/0x7fe2323adfd53f40 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf25 expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074776.497748:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:0.0:1769074776.497772:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616883200:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9086us (9536us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074776.497779:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14157 00000100:00100000:2.0:1769074776.506545:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506595:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506626:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506657:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506674:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506691:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506732:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.506767:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883328 00000100:00100000:2.0:1769074776.506780:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883328 00000100:00100000:2.0:1769074776.506862:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883584 00000100:00100000:2.0:1769074776.506866:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883584 00000100:00100000:2.0:1769074776.506884:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883840 00000100:00100000:2.0:1769074776.506887:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883840 00000100:00100000:2.0:1769074776.506899:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883968 00000100:00100000:2.0:1769074776.506901:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883968 00000100:00100000:2.0:1769074776.506913:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884096 00000100:00100000:2.0:1769074776.506915:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884096 00000100:00100000:2.0:1769074776.506934:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14158 00000100:00100000:2.0:1769074776.506943:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616883328:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.506962:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1769074776.507199:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14159 00000100:00100000:0.0:1769074776.507210:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2408:x1855007616883584:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074776.507219:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074776.507234:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53f4e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf33 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074776.507304:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625157376:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074776.507388:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.507410:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14160 00000100:00100000:0.0:1769074776.507419:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616883840:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074776.507427:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074776.507436:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd53f55 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf3a expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074776.507466:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625157504:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074776.507493:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6466dbb40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074776.509116:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14161 00000100:00100000:1.0:1769074776.509135:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616883968:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074776.509150:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074776.509165:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd53f5c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf41 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.509187:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.509197:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd53f5c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf41 expref: 34 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.509216:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd53f5c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf41 expref: 34 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.509227:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:1.0:1769074776.509306:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616883968:12345-192.168.204.60@tcp:101:cat.0 Request processed in 179us (2613us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074776.509314:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14161 00000100:00100000:1.0:1769074776.509338:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14162 00000100:00100000:1.0:1769074776.509346:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+33:2408:x1855007616884096:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074776.509353:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074776.509360:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd53f63 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf48 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.509380:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.509387:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd53f63 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf48 expref: 34 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.509397:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd53f63 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf48 expref: 34 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.509406:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:1.0:1769074776.509426:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+33:2408:x1855007616884096:12345-192.168.204.60@tcp:101:cat.0 Request processed in 80us (2694us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074776.509432:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14162 00000100:00100000:1.0:1769074776.509459:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883456 00000100:00100000:1.0:1769074776.509465:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883456 00010000:00010000:1.0:1769074776.509488:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074776.509517:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616883712 00000100:00100000:1.0:1769074776.509521:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616883712 00010000:00010000:1.0:1769074776.509533:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074776.509552:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7077 00000100:00100000:1.0:1769074776.509559:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616883456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074776.509571:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074776.509604:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.509619:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.509633:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ba00/0x7fe2323adfd53ebb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed8 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.509642:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074776.509662:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2406:x1855007616883456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 104us (3067us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074776.509668:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7077 00000100:00100000:1.0:1769074776.509711:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7078 00000100:00100000:1.0:1769074776.509716:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+33:2406:x1855007616883712:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074776.509723:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074776.509730:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.509739:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.509750:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773ac00/0x7fe2323adfd53eb4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bed1 expref: 33 pid: 8398 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.509757:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.509759:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.509765:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.509774:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.509799:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074776.509850:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625157632:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074776.509887:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00010000:00010000:2.0:1769074776.510260:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f47 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf2c expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.510301:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625157760:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074776.510334:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074776.513292:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625157504:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074776.513329:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd53f55 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf3a expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.513351:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549e46800/0x7fe2323adfd53f55 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf3a expref: 28 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.513365:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46800, rc 0) 00000100:00100000:3.0:1769074776.513431:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616883840:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6011us (6754us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074776.513440:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14160 00002000:00010000:0.0:1769074776.513640:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime: 1769074775 -> 1769074776 00002000:00010000:0.0:1769074776.513644:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime: 1769074775 -> 1769074776 00002000:00010000:0.0:1769074776.513647:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime: 1769074775 -> 1769074776 00000100:00100000:0.0:1769074776.513659:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625157376:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074776.513676:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53f4e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf33 expref: 32 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074776.513689:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549e47c00/0x7fe2323adfd53f4e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf33 expref: 32 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074776.513699:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47c00, rc 0) 00000100:00100000:0.0:1769074776.513725:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2408:x1855007616883584:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6517us (7066us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074776.513732:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14159 00000100:00100000:1.0:1769074776.515299:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625157632:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074776.515327:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074776.515393:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+30:2406:x1855007616883712:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5674us (8764us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074776.515403:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7078 00000100:00100000:3.0:1769074776.515876:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.515985:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884224 00000100:00100000:1.0:1769074776.515995:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884224 00002000:00100000:1.0:1769074776.516040:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2150400->2154495) req@ffff89b6656d8700 x1855007616884224/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.516050:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d8700 with x1855007616884224 ext(2150400->2154495) 00002000:00010000:1.0:1769074776.516055:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6656d8700 00000100:00100000:1.0:1769074776.516099:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8625 00000100:00100000:1.0:1769074776.516108:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616884224:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074776.516269:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616884224:12345-192.168.204.60@tcp:3:cat.0 Request processed in 162us (394us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074776.516276:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8625 00002000:00100000:1.0:1769074776.516283:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2150400->2154495) req@ffff89b6656d8700 x1855007616884224/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/4536 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.516296:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d8700 with x1855007616884224 ext(2150400->2154495) 00002000:00010000:1.0:1769074776.516299:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b6656d8700 00002000:00010000:2.0:1769074776.516870:0:8396:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime: 1769074775 -> 1769074776 00002000:00010000:2.0:1769074776.516874:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime: 1769074775 -> 1769074776 00002000:00010000:2.0:1769074776.516876:0:8396:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime: 1769074775 -> 1769074776 00000100:00100000:2.0:1769074776.516889:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625157760:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074776.516917:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f47 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf2c expref: 22 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074776.516934:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f47 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf2c expref: 22 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074776.516944:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773a200, rc 0) 00000100:00100000:2.0:1769074776.516991:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2408:x1855007616883328:12345-192.168.204.60@tcp:101:cat.0 Request processed in 10049us (10458us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074776.516999:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14158 00000100:00100000:0.0:1769074776.542173:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.542283:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884352 00000100:00100000:3.0:1769074776.542294:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884352 00002000:00100000:3.0:1769074776.542345:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54385b480 x1855007616884352/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 504/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.542357:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b480 with x1855007616884352 ext(1048576->2097151) 00002000:00010000:3.0:1769074776.542363:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385b480 00000100:00100000:3.0:1769074776.542399:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8626 00000100:00100000:3.0:1769074776.542409:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616884352:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074776.544605:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+21:2406:x1855007616884352:12345-192.168.204.60@tcp:3:cat.0 Request processed in 2181us (2427us total) trans 0 rc 40960/40960 00000100:00100000:3.0:1769074776.544622:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8626 00002000:00100000:3.0:1769074776.544634:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54385b480 x1855007616884352/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 504/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 40960/40960 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.544644:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b480 with x1855007616884352 ext(1048576->2097151) 00002000:00010000:3.0:1769074776.544649:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385b480 00000100:00100000:0.0:1769074776.556100:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.556208:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884480 00000100:00100000:2.0:1769074776.556219:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884480 00000100:00100000:2.0:1769074776.556286:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14163 00000100:00100000:2.0:1769074776.556296:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+21:62958:x1855007616884480:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.556314:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.556335:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf4f expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.556365:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 26 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.556374:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.556376:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.556380:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.556413:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 26 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.556431:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 26 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.556445:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625157888:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.556533:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.561041:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625157888:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074776.561105:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.561129:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773a200, rc 0) 00000100:00100000:2.0:1769074776.561240:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+22:62958:x1855007616884480:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4943us (5147us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.561259:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14163 00000100:00100000:1.0:1769074776.563070:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.563848:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884608 00000100:00100000:0.0:1769074776.563864:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884608 00002000:00100000:0.0:1769074776.563939:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1441792->2097151) req@ffff89b5436d7100 x1855007616884608/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.563952:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7100 with x1855007616884608 ext(1441792->2097151) 00010000:00010000:0.0:1769074776.563962:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 27 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.563979:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 27 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:0.0:1769074776.563988:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5436d7100 00000100:00100000:0.0:1769074776.564018:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 797 00000100:00100000:0.0:1769074776.564029:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616884608:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074776.587457:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616884608:12345-192.168.204.60@tcp:4:cat.0 Request processed in 23417us (24395us total) trans 21474841204 rc 0/0 00000100:00100000:0.0:1769074776.587471:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 797 00002000:00100000:0.0:1769074776.587485:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1441792->2097151) req@ffff89b5436d7100 x1855007616884608/t21474841204(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/448 e 0 to 0 dl 1769074787 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.587496:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7100 with x1855007616884608 ext(1441792->2097151) 00010000:00010000:0.0:1769074776.587507:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 29 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.587525:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 29 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:0.0:1769074776.587533:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5436d7100 00000001:00080000:2.0:1769074776.591259:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841204 is committed 00000001:00080000:2.0:1769074776.591287:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841203 is committed 00000100:00100000:2.0:1769074776.592653:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.593471:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884736 00000100:00100000:1.0:1769074776.593481:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884736 00010000:00010000:1.0:1769074776.593523:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 15 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074776.593685:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7079 00000100:00100000:1.0:1769074776.593696:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616884736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074776.593712:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074776.593732:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 15 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.593746:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 15 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.593768:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7200/0x7fe2323adfd53ec2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1441792->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bedf expref: 15 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.593783:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.593785:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.593792:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.593802:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.593874:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074776.593888:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625158016:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074776.593979:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.599539:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625158016:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074776.599568:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074776.599618:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007616884736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5924us (6974us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.599630:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7079 00000100:00100000:2.0:1769074776.608030:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.608272:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884864 00000100:00100000:0.0:1769074776.608282:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884864 00002000:00100000:0.0:1769074776.608334:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5499d0700 x1855007616884864/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.608345:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0700 with x1855007616884864 ext(1048576->2097151) 00002000:00010000:0.0:1769074776.608351:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5499d0700 00000100:00100000:0.0:1769074776.608390:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8627 00000100:00100000:0.0:1769074776.608670:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616884864:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074776.629233:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616884864:12345-192.168.204.60@tcp:3:cat.0 Request processed in 15179us (15559us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074776.629453:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8627 00002000:00100000:3.0:1769074776.629471:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5499d0700 x1855007616884864/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.629482:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0700 with x1855007616884864 ext(1048576->2097151) 00002000:00010000:3.0:1769074776.629487:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5499d0700 00000100:00100000:1.0:1769074776.648830:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.651530:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616884992 00000100:00100000:2.0:1769074776.651548:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616884992 00000100:00100000:2.0:1769074776.651650:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14164 00000100:00100000:2.0:1769074776.651661:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+30:62958:x1855007616884992:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.651683:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.651707:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf56 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.651739:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.651749:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.651751:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.651756:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.651792:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 31 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.651848:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 31 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.651863:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625158144:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.651957:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.655416:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625158144:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074776.655505:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.655524:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773be00, rc 0) 00000100:00100000:3.0:1769074776.657839:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.657944:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.658006:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616885248 00000100:00100000:3.0:1769074776.658017:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616885248 00000100:00100000:3.0:1769074776.658076:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13334 00000100:00100000:3.0:1769074776.658086:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+249:62964:x1855007616885248:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.658114:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.658140:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53f78 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf5d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.658222:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658254:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658356:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658376:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658384:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658391:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658520:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53f8d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658534:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53f8d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658542:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53f8d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074776.658553:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074776.658636:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658644:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658649:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658655:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658662:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fea4000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074776.658666:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658673:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658684:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074776.658686:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074776.658691:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658696:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4000/0x7fe2323adfd53f7f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658711:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658716:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658721:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658726:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b446200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074776.658729:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658734:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658741:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6439ad800) 00010000:00010000:3.0:1769074776.658743:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074776.658746:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658752:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd53f86 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074776.658792:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4a00/0x7fe2323adfd53f78 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf5d expref: 252 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.658854:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53f8d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf5d expref: 251 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.658861:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074776.658863:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074776.658866:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446800, rc 0) 00000100:00100000:3.0:1769074776.658919:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62964:x1855007616885248:12345-192.168.204.60@tcp:101:cat.0 Request processed in 834us (974us total) trans 8589943975 rc 0/0 00000100:00100000:3.0:1769074776.658927:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13334 00000100:00100000:3.0:1769074776.658959:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616885376 00000100:00100000:3.0:1769074776.658964:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616885376 00002000:00100000:3.0:1769074776.658991:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b6656d8000 x1855007616885376/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 504/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.659000:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d8000 with x1855007616885376 ext(1048576->2097151) 00010000:00010000:3.0:1769074776.659008:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 32 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:3.0:1769074776.659021:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 32 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:3.0:1769074776.659034:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b6656d8000 00000100:00100000:3.0:1769074776.659054:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 798 00000100:00100000:3.0:1769074776.659061:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616885376:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074776.659962:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62958:x1855007616884992:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8298us (11159us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.659971:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14164 00000100:00100000:3.0:1769074776.672237:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.672500:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.672789:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616885760 00000100:00100000:0.0:1769074776.672855:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616885760 00000100:00100000:0.0:1769074776.672937:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1024 00000100:00100000:0.0:1769074776.672949:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:2406:x1855007616885760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074776.672972:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074776.673008:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446800/0x7fe2323adfd53f8d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf5d expref: 251 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074776.673036:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074776.673042:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074776.673114:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+250:2406:x1855007616885760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 167us (613us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074776.673122:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1024 00000100:00100000:1.0:1769074776.673431:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616885632 00000100:00100000:1.0:1769074776.673439:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616885632 00000100:00100000:1.0:1769074776.673482:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4682 00000100:00100000:1.0:1769074776.673491:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+250:62964:x1855007616885632:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:1.0:1769074776.673777:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62964:x1855007616885632:12345-192.168.204.60@tcp:35:cat.0 Request processed in 287us (1552us total) trans 8589943976 rc 0/0 00000100:00100000:1.0:1769074776.673784:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4682 00000100:00100000:3.0:1769074776.682530:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.682563:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.682635:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.683230:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2408:x1855007616885376:12345-192.168.204.60@tcp:4:cat.0 Request processed in 24166us (25431us total) trans 17179874524 rc 0/0 00000100:00100000:2.0:1769074776.683253:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 798 00002000:00100000:2.0:1769074776.683272:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b6656d8000 x1855007616885376/t17179874524(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 504/448 e 0 to 0 dl 1769074787 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.683290:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b6656d8000 with x1855007616885376 ext(1048576->2097151) 00010000:00010000:2.0:1769074776.683300:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 34 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.683316:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 34 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:2.0:1769074776.683324:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b6656d8000 00000100:00100000:3.0:1769074776.683842:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.684063:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616885888 00000100:00100000:1.0:1769074776.684072:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616885888 00000100:00100000:1.0:1769074776.684197:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886016 00000100:00100000:1.0:1769074776.684202:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886016 00000100:00100000:1.0:1769074776.684219:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886144 00000100:00100000:1.0:1769074776.684221:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886144 00000100:00100000:1.0:1769074776.684252:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14165 00000100:00100000:1.0:1769074776.684261:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616885888:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074776.684355:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074776.684373:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53f9b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf64 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074776.684502:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625158272:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074776.684568:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074776.684773:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886272 00000100:00100000:3.0:1769074776.684778:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886272 00000100:00100000:1.0:1769074776.684828:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14166 00000100:00100000:1.0:1769074776.684836:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616886016:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074776.684844:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074776.684852:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf6b expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074776.684864:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.684866:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074776.684874:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 17 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.684889:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 17 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074776.684900:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651e00, rc 0) 00000100:00100000:1.0:1769074776.684923:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616886016:12345-192.168.204.60@tcp:101:cat.0 Request processed in 88us (2360us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.684981:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14167 00000100:00100000:3.0:1769074776.684988:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616886144:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.684995:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.685005:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53fa9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf72 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074776.685032:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625158400:192.168.204.60@tcp:106: 00000100:00100000:3.0:1769074776.685060:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5b0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074776.685074:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14168 00000100:00100000:3.0:1769074776.685080:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616886272:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.685086:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.685094:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd53fb0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf79 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.685129:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd53fb0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf79 expref: 37 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.685143:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd53fb0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf79 expref: 37 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.685154:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446800, rc 0) 00000100:00100000:3.0:1769074776.685171:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616886272:12345-192.168.204.60@tcp:101:cat.0 Request processed in 91us (1332us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074776.685177:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14168 00000100:00100000:3.0:1769074776.687170:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625158400:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074776.687205:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53fa9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf72 expref: 35 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.687228:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67602b800/0x7fe2323adfd53fa9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf72 expref: 35 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.687252:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602b800, rc 0) 00000100:00100000:3.0:1769074776.687311:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2409:x1855007616886144:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2322us (4674us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074776.687321:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14167 00002000:00010000:3.0:1769074776.687367:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime: 1769074775 -> 1769074776 00000100:00100000:3.0:1769074776.687379:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625158272:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074776.687409:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53f9b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf64 expref: 17 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.687423:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651a00/0x7fe2323adfd53f9b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf64 expref: 17 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.687432:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641651a00, rc 0) 00000100:00100000:3.0:1769074776.687453:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616885888:12345-192.168.204.60@tcp:101:cat.0 Request processed in 3193us (4932us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074776.687459:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14165 00000100:00100000:3.0:1769074776.688893:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.689096:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886400 00000100:00100000:2.0:1769074776.689115:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886400 00010000:00010000:2.0:1769074776.689153:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 34 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.689193:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7080 00000100:00100000:2.0:1769074776.689202:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2407:x1855007616886400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074776.689215:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074776.689233:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 34 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.689255:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 34 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.689277:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773a000/0x7fe2323adfd53ede lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1273855) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879befb expref: 34 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.689298:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.689300:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.689308:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 30 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.689319:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 30 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.689353:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 30 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.689367:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625158528:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074776.689450:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074776.690344:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14166 00000100:00100000:3.0:1769074776.698793:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625158528:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074776.698858:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074776.698939:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2407:x1855007616886400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9737us (10056us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.698950:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7080 00000100:00100000:2.0:1769074776.704799:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.704930:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886528 00000100:00100000:0.0:1769074776.704941:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886528 00002000:00100000:0.0:1769074776.704997:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5499d2680 x1855007616886528/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.705014:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2680 with x1855007616886528 ext(1048576->2097151) 00002000:00010000:0.0:1769074776.705019:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d2680 00000100:00100000:0.0:1769074776.705098:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8628 00000100:00100000:0.0:1769074776.705119:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2406:x1855007616886528:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074776.716431:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.718760:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886656 00000100:00100000:2.0:1769074776.718777:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886656 00000100:00100000:2.0:1769074776.718903:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14169 00000100:00100000:2.0:1769074776.718913:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:62964:x1855007616886656:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.718932:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.718952:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf80 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.718993:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 9 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.719004:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.719010:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074776.719014:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.719053:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 9 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.719068:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 9 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.719084:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625158656:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074776.719192:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.723118:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+31:2406:x1855007616886528:12345-192.168.204.60@tcp:3:cat.0 Request processed in 17987us (18311us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074776.723135:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8628 00002000:00100000:0.0:1769074776.723150:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5499d2680 x1855007616886528/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.723161:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2680 with x1855007616886528 ext(1048576->2097151) 00002000:00010000:0.0:1769074776.723168:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d2680 00000100:00100000:2.0:1769074776.724377:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625158656:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074776.724424:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 16 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.724441:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e400, rc 0) 00000100:00100000:2.0:1769074776.724498:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:62964:x1855007616886656:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5585us (8079us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.724506:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14169 00000100:00100000:3.0:1769074776.731036:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.731574:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886784 00000100:00100000:0.0:1769074776.731586:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886784 00002000:00100000:0.0:1769074776.731637:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1081344->2088959) req@ffff89b5437b5c00 x1855007616886784/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.731649:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5c00 with x1855007616886784 ext(1081344->2088959) 00010000:00010000:0.0:1769074776.731659:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 10 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.731676:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 10 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:0.0:1769074776.731690:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b5c00 00000100:00100000:0.0:1769074776.731732:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 799 00000100:00100000:0.0:1769074776.731743:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616886784:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074776.741714:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.741771:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616886912 00000100:00100000:3.0:1769074776.741791:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616886912 00000100:00100000:3.0:1769074776.741889:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14170 00000100:00100000:3.0:1769074776.741902:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62958:x1855007616886912:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.741927:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.741946:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf87 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.741977:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 32 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.741992:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074776.741994:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074776.742000:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.742047:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 32 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.742064:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 32 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074776.742081:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625158784:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074776.744774:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5b0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.755411:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616886784:12345-192.168.204.60@tcp:4:cat.0 Request processed in 23657us (24378us total) trans 21474841205 rc 0/0 00000100:00100000:0.0:1769074776.755426:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 799 00002000:00100000:0.0:1769074776.755439:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1081344->2088959) req@ffff89b5437b5c00 x1855007616886784/t21474841205(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/448 e 0 to 0 dl 1769074787 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.755450:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5437b5c00 with x1855007616886784 ext(1081344->2088959) 00010000:00010000:0.0:1769074776.755460:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 12 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074776.755477:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 12 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:0.0:1769074776.755485:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b5437b5c00 00000100:00100000:0.0:1769074776.758651:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.758848:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887040 00000100:00100000:1.0:1769074776.758859:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887040 00002000:00100000:1.0:1769074776.758909:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2154495) req@ffff89b54385b800 x1855007616887040/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.758922:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b800 with x1855007616887040 ext(2097152->2154495) 00010000:00010000:1.0:1769074776.758931:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 33 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.758946:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 33 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074776.758953:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54385b800 00000100:00100000:1.0:1769074776.758995:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 800 00000100:00100000:1.0:1769074776.759005:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2407:x1855007616887040:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074776.759950:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625158784:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074776.759981:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.759997:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446200, rc 0) 00000100:00100000:3.0:1769074776.760055:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62958:x1855007616886912:12345-192.168.204.60@tcp:101:cat.0 Request processed in 18154us (18355us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.760064:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14170 00000100:00100000:2.0:1769074776.770557:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.775985:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2407:x1855007616887040:12345-192.168.204.60@tcp:4:cat.0 Request processed in 16968us (17331us total) trans 17179874525 rc 0/0 00000100:00100000:1.0:1769074776.776002:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 800 00002000:00100000:1.0:1769074776.776016:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2154495) req@ffff89b54385b800 x1855007616887040/t17179874525(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/448 e 0 to 0 dl 1769074787 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.776027:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b800 with x1855007616887040 ext(2097152->2154495) 00010000:00010000:1.0:1769074776.776037:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 35 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074776.776052:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 35 pid: 9048 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074776.776060:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54385b800 00000100:00100000:3.0:1769074776.780901:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887168 00000100:00100000:3.0:1769074776.780917:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887168 00010000:00010000:2.0:1769074776.781061:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 12 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.781156:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7081 00000100:00100000:2.0:1769074776.781167:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616887168:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074776.781183:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074776.781204:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 12 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.781217:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 12 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.781248:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773b800/0x7fe2323adfd53f08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->2088959) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf09 expref: 12 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.781273:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.781275:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.781283:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.781293:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.781331:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.781345:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625158912:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074776.781435:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074776.783621:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.783954:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887296 00000100:00100000:3.0:1769074776.783968:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887296 00010000:00010000:2.0:1769074776.784091:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 35 pid: 9048 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074776.784488:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7082 00000100:00100000:2.0:1769074776.784500:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2406:x1855007616887296:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074776.784516:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1769074776.784535:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074775 -> 1769074776 00010000:00010000:2.0:1769074776.784542:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 35 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.784557:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 35 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.784579:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea5400/0x7fe2323adfd53f39 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->2883583] (req 2097152->2154495) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf1e expref: 35 pid: 9048 timeout: 1864 lvb_type: 0 00010000:00010000:2.0:1769074776.784610:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.784612:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.784620:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.784629:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.784666:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074776.784680:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625159040:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074776.784831:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074776.791106:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625158912:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074776.791161:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074776.791226:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616887168:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10058us (20679us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.791253:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7081 00000100:00100000:2.0:1769074776.795016:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625159040:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074776.795046:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074776.795118:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616887296:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10620us (11504us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.795398:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7082 00000100:00100000:0.0:1769074776.804673:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.804782:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887424 00000100:00100000:1.0:1769074776.804794:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887424 00002000:00100000:1.0:1769074776.804901:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b543858e00 x1855007616887424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.804920:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858e00 with x1855007616887424 ext(1048576->2097151) 00002000:00010000:1.0:1769074776.804927:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543858e00 00000100:00100000:1.0:1769074776.804966:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8629 00000100:00100000:1.0:1769074776.804976:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616887424:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074776.807316:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.807376:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.807417:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887552 00000100:00100000:3.0:1769074776.807426:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887552 00002000:00100000:3.0:1769074776.807467:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67b6d7450 x1855007616887552/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.807478:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b6d7450 with x1855007616887552 ext(1048576->2097151) 00002000:00010000:3.0:1769074776.807484:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b6d7450 00000100:00100000:3.0:1769074776.807510:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8630 00000100:00100000:3.0:1769074776.807518:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b6d7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616887552:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074776.807748:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887808 00000100:00100000:2.0:1769074776.807753:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887808 00000100:00100000:2.0:1769074776.807794:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13335 00000100:00100000:2.0:1769074776.807801:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+251:62968:x1855007616887808:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074776.807834:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.807854:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd53fc5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf8e expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.807945:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.807960:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.807971:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.807987:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.807993:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.807999:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808123:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fda lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808135:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fda lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808143:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fda lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074776.808153:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:2.0:1769074776.808382:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808390:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808397:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808405:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808410:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074776.808414:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808423:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808437:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074776.808440:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074776.808445:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808452:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd53fcc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808468:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808475:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808481:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808486:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074776.808490:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808496:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808503:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6436c4a00) 00010000:00010000:2.0:1769074776.808505:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074776.808508:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808514:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd53fd3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074776.808560:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd53fc5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf8e expref: 254 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.808575:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fda lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf8e expref: 253 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.808583:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074776.808586:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074776.808589:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:2.0:1769074776.808630:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62968:x1855007616887808:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 829us (1326us total) trans 8589943977 rc 0/0 00000100:00100000:2.0:1769074776.808641:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13335 00000100:00100000:3.0:1769074776.816176:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.825372:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616887936 00000100:00100000:0.0:1769074776.825392:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616887936 00002000:00100000:0.0:1769074776.827018:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2883583) req@ffff89b67a272300 x1855007616887936/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074776.827037:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a272300 with x1855007616887936 ext(2097152->2883583) 00002000:00010000:0.0:1769074776.827046:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a272300 00000100:00100000:0.0:1769074776.827125:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8631 00000100:00100000:0.0:1769074776.827135:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2406:x1855007616887936:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074776.840865:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074776.840933:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888192 00000100:00100000:0.0:1769074776.840944:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888192 00000100:00100000:0.0:1769074776.841002:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4683 00000100:00100000:0.0:1769074776.841013:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62968:x1855007616888192:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074776.841880:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.841939:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888320 00000100:00100000:3.0:1769074776.841945:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888320 00000100:00100000:3.0:1769074776.841983:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1025 00000100:00100000:3.0:1769074776.841990:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e89d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:2408:x1855007616888320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074776.842002:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074776.842025:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fda lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf8e expref: 255 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.842043:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074776.842048:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074776.842170:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e89d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+254:2408:x1855007616888320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 179us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.842178:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1025 00000100:00100000:3.0:1769074776.843053:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2406:x1855007616887936:12345-192.168.204.60@tcp:3:cat.0 Request processed in 15913us (26890us total) trans 0 rc 786432/786432 00000100:00100000:3.0:1769074776.843206:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8631 00002000:00100000:3.0:1769074776.843220:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2883583) req@ffff89b67a272300 x1855007616887936/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 786432/786432 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074776.843231:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a272300 with x1855007616887936 ext(2097152->2883583) 00002000:00010000:3.0:1769074776.843249:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a272300 00000100:00100000:0.0:1769074776.845483:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62968:x1855007616888192:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 4459us (4617us total) trans 8589943978 rc 0/0 00000100:00100000:0.0:1769074776.845498:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4683 00000100:00100000:2.0:1769074776.850601:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b6d7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616887552:12345-192.168.204.60@tcp:3:cat.0 Request processed in 43067us (43208us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074776.850616:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8630 00002000:00100000:2.0:1769074776.850632:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b67b6d7450 x1855007616887552/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074776.850642:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67b6d7450 with x1855007616887552 ext(1048576->2097151) 00002000:00010000:2.0:1769074776.850647:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67b6d7450 00000100:00100000:1.0:1769074776.851938:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616887424:12345-192.168.204.60@tcp:3:cat.0 Request processed in 46947us (47265us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074776.851954:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8629 00002000:00100000:1.0:1769074776.851971:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b543858e00 x1855007616887424/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.851982:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858e00 with x1855007616887424 ext(1048576->2097151) 00002000:00010000:1.0:1769074776.851989:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543858e00 00000100:00100000:2.0:1769074776.864358:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074776.864521:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888448 00000100:00100000:3.0:1769074776.864533:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888448 00000100:00100000:3.0:1769074776.865581:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14171 00000100:00100000:3.0:1769074776.865790:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616888448:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074776.865857:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074776.865882:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf95 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074776.865913:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074776.865916:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074776.865929:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.865951:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074776.865968:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:3.0:1769074776.866052:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616888448:12345-192.168.204.60@tcp:101:cat.0 Request processed in 271us (1703us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074776.875160:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14171 00000100:00100000:3.0:1769074776.878217:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.878368:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888576 00000100:00100000:2.0:1769074776.878381:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888576 00000100:00100000:2.0:1769074776.878453:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14172 00000100:00100000:2.0:1769074776.878465:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:62964:x1855007616888576:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.878484:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.878506:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bf9c expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.878540:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.878543:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.878554:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.878569:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.878582:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:2.0:1769074776.878656:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:62964:x1855007616888576:12345-192.168.204.60@tcp:101:cat.0 Request processed in 193us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.878664:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14172 00000100:00100000:0.0:1769074776.897583:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.900466:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888704 00000100:00100000:1.0:1769074776.900485:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888704 00002000:00100000:1.0:1769074776.900559:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b543859500 x1855007616888704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.900571:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859500 with x1855007616888704 ext(1048576->2097151) 00002000:00010000:1.0:1769074776.900577:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543859500 00000100:00100000:1.0:1769074776.900616:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8632 00000100:00100000:1.0:1769074776.900626:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2408:x1855007616888704:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074776.914915:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2408:x1855007616888704:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14271us (17327us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074776.914934:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8632 00002000:00100000:1.0:1769074776.914954:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b543859500 x1855007616888704/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.914967:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859500 with x1855007616888704 ext(1048576->2097151) 00002000:00010000:1.0:1769074776.914974:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543859500 00000100:00100000:3.0:1769074776.941954:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074776.942117:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888832 00000100:00100000:2.0:1769074776.942128:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888832 00000100:00100000:2.0:1769074776.942183:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14173 00000100:00100000:2.0:1769074776.942193:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:62964:x1855007616888832:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074776.942212:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074776.942232:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfa3 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.942280:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.942284:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074776.942297:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 36 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.942317:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 36 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074776.942332:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:2.0:1769074776.942409:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:62964:x1855007616888832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 216us (470us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074776.942416:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14173 00000100:00100000:0.0:1769074776.954204:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074776.954306:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616888960 00000100:00100000:1.0:1769074776.954320:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616888960 00002000:00100000:1.0:1769074776.954381:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2883583) req@ffff89b54385b100 x1855007616888960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/0 e 0 to 0 dl 1769074787 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.954609:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b100 with x1855007616888960 ext(2097152->2883583) 00002000:00010000:1.0:1769074776.954617:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54385b100 00000100:00100000:1.0:1769074776.954724:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8633 00000100:00100000:1.0:1769074776.954740:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2408:x1855007616888960:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074776.968514:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2408:x1855007616888960:12345-192.168.204.60@tcp:3:cat.0 Request processed in 13768us (14314us total) trans 0 rc 786432/786432 00000100:00100000:1.0:1769074776.968529:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8633 00002000:00100000:1.0:1769074776.968541:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->2883583) req@ffff89b54385b100 x1855007616888960/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:312/0 lens 488/440 e 0 to 0 dl 1769074787 ref 1 fl Complete:/600/0 rc 786432/786432 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074776.968552:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b100 with x1855007616888960 ext(2097152->2883583) 00002000:00010000:1.0:1769074776.968559:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54385b100 00000100:00100000:3.0:1769074777.055925:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.056316:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889088 00000100:00100000:1.0:1769074777.056337:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889088 00002000:00100000:1.0:1769074777.056437:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54ce9e300 x1855007616889088/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.056449:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9e300 with x1855007616889088 ext(1048576->2097151) 00002000:00010000:1.0:1769074777.056460:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9e300 00000100:00100000:1.0:1769074777.056512:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8634 00000100:00100000:1.0:1769074777.056525:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616889088:12345-192.168.204.60@tcp:3:cat.0 00000100:00080000:2.0:1769074777.056743:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074777.065100:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.065446:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889216 00000100:00100000:1.0:1769074777.065460:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889216 00000100:00100000:1.0:1769074777.065553:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14174 00000100:00100000:1.0:1769074777.065566:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:62968:x1855007616889216:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074777.065602:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074777.065626:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfaa expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.065656:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.065665:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 37 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.065672:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.065674:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.065678:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [1069056->1908735] (req 1069056->1908735) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.065717:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 37 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.065732:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 37 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.065750:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625159168:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074777.065897:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.065907:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 31 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.065916:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625159296:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074777.065943:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00080000:2.0:1769074777.066740:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074777.067626:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.067917:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616889088:12345-192.168.204.60@tcp:3:cat.0 Request processed in 11393us (12004us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074777.067930:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8634 00002000:00100000:1.0:1769074777.067941:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54ce9e300 x1855007616889088/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.067950:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9e300 with x1855007616889088 ext(1048576->2097151) 00002000:00010000:1.0:1769074777.067956:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9e300 00000100:00100000:3.0:1769074777.068661:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889344 00000100:00100000:3.0:1769074777.068670:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889344 00000100:00100000:3.0:1769074777.068759:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14175 00000100:00100000:3.0:1769074777.068768:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62966:x1855007616889344:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074777.068784:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074777.068800:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfb1 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.068845:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.068854:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.068860:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074777.068862:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074777.068866:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2928640->3145727] (req 2928640->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.068895:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.068905:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 15 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074777.068923:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625159424:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074777.069000:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.069008:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 13 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074777.069016:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625159552:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074777.069037:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.071383:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.071421:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.071951:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889600 00000100:00100000:0.0:1769074777.071961:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889600 00010000:00010000:0.0:1769074777.072024:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:0.0:1769074777.072064:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889472 00000100:00100000:0.0:1769074777.072067:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889472 00010000:00010000:0.0:1769074777.072082:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 38 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:0.0:1769074777.072105:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7083 00000100:00100000:0.0:1769074777.072114:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2409:x1855007616889600:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074777.072130:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074777.072156:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.072168:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.072198:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074777.072273:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2409:x1855007616889600:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 159us (901us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.072279:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7083 00000100:00100000:0.0:1769074777.072297:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7084 00000100:00100000:0.0:1769074777.072303:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2409:x1855007616889472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074777.072310:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074777.072317:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 38 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.072327:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 38 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.072340:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.072342:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.072350:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 38 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.072359:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 38 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.072416:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 38 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074777.072431:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625159680:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074777.072480:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6466dbb40 going to sleep for 11 seconds 00010000:00010000:1.0:1769074777.073016:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd53fef lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf9c expref: 38 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.073034:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625159168:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074777.073048:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773be00/0x7fe2323adfd53f71 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf56 expref: 31 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.073064:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625159296:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074777.073094:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 37 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.073105:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650a00, rc 0) 00000100:00100000:1.0:1769074777.073156:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:62968:x1855007616889216:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 7594us (8070us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.073165:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14174 00000100:00080000:2.0:1769074777.073556:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074777.080781:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.080868:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.080968:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889728 00000100:00100000:0.0:1769074777.080979:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889728 00010000:00010000:0.0:1769074777.081033:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 16 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:0.0:1769074777.081075:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889856 00000100:00100000:0.0:1769074777.081078:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889856 00010000:00010000:0.0:1769074777.081093:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 14 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:0.0:1769074777.081118:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7085 00000100:00100000:0.0:1769074777.081129:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616889728:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074777.081149:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074777.081175:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 16 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.081191:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 16 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.081220:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074777.081296:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616889728:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 168us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.081305:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7085 00000100:00100000:0.0:1769074777.081327:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7086 00000100:00100000:0.0:1769074777.081335:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616889856:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074777.081343:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074777.081353:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 14 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.081364:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 14 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:0.0:1769074777.081379:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.081382:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.081405:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.081419:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.081467:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074777.081483:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625159808:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074777.081543:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00010000:00010000:3.0:1769074777.082353:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773bc00/0x7fe2323adfd53d09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdce expref: 15 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074777.082375:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625159424:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074777.082391:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b679924a00/0x7fe2323adfd53d02 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bdc7 expref: 14 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074777.082400:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625159552:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074777.082431:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.082451:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876c00, rc 0) 00000100:00100000:3.0:1769074777.082506:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62966:x1855007616889344:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 13738us (14887us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074777.082518:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14175 00000100:00100000:1.0:1769074777.083722:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625159680:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074777.083744:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.083794:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616889472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11489us (12370us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.083837:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7084 00000100:00100000:0.0:1769074777.086636:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.086679:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.086745:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890112 00000100:00100000:3.0:1769074777.086756:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890112 00000100:00100000:3.0:1769074777.086835:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14176 00000100:00100000:3.0:1769074777.086844:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+31:62973:x1855007616890112:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074777.086862:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074777.086879:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfb8 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.086905:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 35 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.086912:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074777.086914:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074777.086918:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.086946:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 35 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.086957:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 35 pid: 28954 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074777.086968:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625159936:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074777.087040:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074777.087497:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616889984 00000100:00100000:1.0:1769074777.087504:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616889984 00002000:00100000:1.0:1769074777.087544:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1069056->1073151) req@ffff89b543859c00 x1855007616889984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.087554:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859c00 with x1855007616889984 ext(1069056->1073151) 00010000:00010000:1.0:1769074777.087561:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.087574:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074777.087596:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b543859c00 00000100:00100000:1.0:1769074777.087623:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 801 00000100:00100000:1.0:1769074777.087631:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616889984:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074777.087890:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2409:x1855007616889984:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 258us (1263us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074777.087898:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 801 00002000:00100000:1.0:1769074777.087904:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1069056->1073151) req@ffff89b543859c00 x1855007616889984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.087914:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859c00 with x1855007616889984 ext(1069056->1073151) 00010000:00010000:1.0:1769074777.087919:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.087929:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074777.087936:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b543859c00 00000100:00100000:3.0:1769074777.090051:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6499:1855007625159808:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074777.090075:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074777.090147:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616889856:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8811us (9276us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074777.090159:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7086 00000100:00100000:0.0:1769074777.091151:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.091223:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890240 00000100:00100000:1.0:1769074777.091231:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890240 00002000:00100000:1.0:1769074777.091288:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2928640->2932735) req@ffff89b543859f80 x1855007616890240/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.091298:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859f80 with x1855007616890240 ext(2928640->2932735) 00002000:00010000:1.0:1769074777.091303:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543859f80 00000100:00100000:1.0:1769074777.091366:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8635 00000100:00100000:1.0:1769074777.091377:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616890240:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074777.091387:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625159936:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.091426:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 32 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.091491:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877800, rc 0) 00000100:00100000:1.0:1769074777.091544:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616890240:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 168us (397us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074777.091552:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8635 00002000:00100000:1.0:1769074777.091558:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2928640->2932735) req@ffff89b543859f80 x1855007616890240/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.091567:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859f80 with x1855007616890240 ext(2928640->2932735) 00002000:00010000:1.0:1769074777.091570:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543859f80 00000100:00100000:2.0:1769074777.093221:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62973:x1855007616890112:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6365us (6528us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.093247:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14176 00000100:00100000:0.0:1769074777.097940:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.098369:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890368 00000100:00100000:2.0:1769074777.098382:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890368 00000100:00100000:2.0:1769074777.098456:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14177 00000100:00100000:2.0:1769074777.098468:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:62966:x1855007616890368:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074777.098490:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.098512:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfbf expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098549:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 33 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098562:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098570:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 35 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098577:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098729:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:2.0:1769074777.098731:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:2.0:1769074777.098736:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 7/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [3145728->3862527] (req 3145728->3862527) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 33 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098776:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.098788:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074777.098802:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160064:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.099059:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 35 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.099069:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 35 pid: 27843 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074777.099079:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160192:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.099109:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 35 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.099117:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 35 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074777.099126:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160320:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.099154:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 33 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.099162:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 33 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:2.0:1769074777.099171:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160448:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.099193:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074777.109200:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.109442:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160064:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.109454:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160192:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.109463:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160320:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.109471:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625160448:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.109504:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [3145728->3862527] (req 3145728->3862527) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 33 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.109528:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910d200, rc 0) 00000100:00100000:2.0:1769074777.109705:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+33:62966:x1855007616890368:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 11238us (11773us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.109714:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14177 00000100:00100000:1.0:1769074777.109884:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890496 00000100:00100000:1.0:1769074777.109891:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890496 00002000:00100000:1.0:1769074777.109930:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1904640->1908735) req@ffff89b543858380 x1855007616890496/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.109939:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858380 with x1855007616890496 ext(1904640->1908735) 00010000:00010000:1.0:1769074777.109946:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.109958:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074777.109964:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b543858380 00000100:00100000:1.0:1769074777.109991:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 802 00000100:00100000:1.0:1769074777.109998:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2406:x1855007616890496:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:1.0:1769074777.110365:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2406:x1855007616890496:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 365us (1174us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074777.110377:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 802 00002000:00100000:1.0:1769074777.110383:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1904640->1908735) req@ffff89b543858380 x1855007616890496/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.110391:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858380 with x1855007616890496 ext(1904640->1908735) 00010000:00010000:1.0:1769074777.110397:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.110406:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074777.110412:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b543858380 00000100:00100000:0.0:1769074777.122153:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.122264:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890880 00000100:00100000:1.0:1769074777.122275:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890880 00002000:00100000:1.0:1769074777.122322:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1069056->1908735) req@ffff89b54385a680 x1855007616890880/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.122333:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a680 with x1855007616890880 ext(1069056->1908735) 00010000:00010000:1.0:1769074777.122342:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.122353:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074777.122361:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54385a680 00000100:00100000:1.0:1769074777.122401:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 803 00000100:00100000:1.0:1769074777.122412:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2406:x1855007616890880:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:0.0:1769074777.137025:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.137060:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.137596:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.139132:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.139505:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890752 00000100:00100000:1.0:1769074777.139518:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890752 00010000:00010000:1.0:1769074777.139565:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 38 pid: 27843 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.139857:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891136 00000100:00100000:1.0:1769074777.139861:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891136 00010000:00010000:1.0:1769074777.139877:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 39 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.139901:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616890624 00000100:00100000:1.0:1769074777.139904:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616890624 00010000:00010000:1.0:1769074777.139917:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.139938:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7087 00000100:00100000:1.0:1769074777.139947:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+39:2408:x1855007616890752:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.139964:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.139989:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 39 pid: 27843 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140001:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 39 pid: 27843 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140027:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718600/0x7fe2323adfd53df7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 7 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be53 expref: 39 pid: 27843 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140052:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.140123:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2408:x1855007616890752:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 176us (3108us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.140132:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7087 00000100:00100000:1.0:1769074777.140154:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7088 00000100:00100000:1.0:1769074777.140161:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2408:x1855007616891136:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.140169:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.140176:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 37 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140186:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 37 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140199:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd53ff6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfa3 expref: 37 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140209:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.140231:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2408:x1855007616891136:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 71us (3171us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.140249:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7088 00000100:00100000:1.0:1769074777.140263:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7089 00000100:00100000:1.0:1769074777.140270:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+33:2408:x1855007616890624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.140279:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.140288:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140300:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140314:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54773b600/0x7fe2323adfd53dfe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2883584->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879be4c expref: 33 pid: 36516 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.140323:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.140343:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2408:x1855007616890624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 74us (2762us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.140350:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7089 00000100:00100000:1.0:1769074777.141469:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891008 00000100:00100000:1.0:1769074777.141482:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891008 00010000:00010000:1.0:1769074777.141513:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00000100:00100000:1.0:1769074777.141544:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7090 00000100:00100000:1.0:1769074777.141553:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+32:2408:x1855007616891008:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.141563:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.142056:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.142072:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.142096:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b446200/0x7fe2323adfd53fbe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf87 expref: 32 pid: 8396 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.142114:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.142117:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.142124:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.142135:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.142173:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 31 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074777.142186:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625160576:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074777.142277:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074777.146756:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2406:x1855007616890880:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 24335us (24601us total) trans 17179874526 rc 0/0 00000100:00100000:1.0:1769074777.146773:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 803 00002000:00100000:1.0:1769074777.146787:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2be:0x0] (1069056->1908735) req@ffff89b54385a680 x1855007616890880/t17179874526(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/448 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.146799:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a680 with x1855007616890880 ext(1069056->1908735) 00010000:00010000:1.0:1769074777.146834:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:1.0:1769074777.146854:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 36 pid: 28954 timeout: 1864 lvb_type: 0 00002000:00010000:1.0:1769074777.146868:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54385a680 00000001:00080000:1.0:1769074777.155413:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874526 is committed 00000001:00080000:1.0:1769074777.155451:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874521 is committed 00000100:00100000:0.0:1769074777.157636:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.157956:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891264 00000100:00100000:3.0:1769074777.157967:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891264 00010000:00010000:3.0:1769074777.158007:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 20 pid: 28954 timeout: 1864 lvb_type: 0 00000100:00100000:3.0:1769074777.158057:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7091 00000100:00100000:3.0:1769074777.158067:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616891264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074777.158085:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074777.158107:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 20 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:3.0:1769074777.158119:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 20 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:3.0:1769074777.158142:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd53ffd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1908735) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfaa expref: 20 pid: 28954 timeout: 1864 lvb_type: 0 00010000:00010000:3.0:1769074777.158158:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074777.158160:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074777.158167:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.158176:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.158213:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 9 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074777.158228:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625160704:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074777.158364:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.160135:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.160221:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891392 00000100:00100000:3.0:1769074777.160231:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891392 00002000:00100000:3.0:1769074777.160287:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3858432->3862527) req@ffff89b5499d1180 x1855007616891392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.160299:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1180 with x1855007616891392 ext(3858432->3862527) 00002000:00010000:3.0:1769074777.160305:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d1180 00000100:00100000:3.0:1769074777.160338:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8636 00000100:00100000:3.0:1769074777.160347:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616891392:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:3.0:1769074777.160531:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616891392:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 183us (404us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074777.160539:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8636 00002000:00100000:3.0:1769074777.160546:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3858432->3862527) req@ffff89b5499d1180 x1855007616891392/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.160554:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1180 with x1855007616891392 ext(3858432->3862527) 00002000:00010000:3.0:1769074777.160557:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d1180 00000100:00100000:0.0:1769074777.166841:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625160704:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074777.166874:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074777.166941:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616891264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8875us (9315us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.166952:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7091 00000100:00100000:1.0:1769074777.168923:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625160576:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074777.168959:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.169036:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616891008:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 27481us (29909us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.169048:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7090 00000100:00100000:3.0:1769074777.183218:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.184481:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891520 00000100:00100000:2.0:1769074777.184496:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891520 00002000:00100000:2.0:1769074777.184557:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54ce9ca80 x1855007616891520/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.184570:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ca80 with x1855007616891520 ext(1048576->2097151) 00002000:00010000:2.0:1769074777.184576:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9ca80 00000100:00100000:2.0:1769074777.184623:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8637 00000100:00100000:2.0:1769074777.184632:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616891520:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074777.204116:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616891520:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 19473us (20895us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074777.204130:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8637 00002000:00100000:2.0:1769074777.204143:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54ce9ca80 x1855007616891520/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.204154:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9ca80 with x1855007616891520 ext(1048576->2097151) 00002000:00010000:2.0:1769074777.204161:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9ca80 00000100:00100000:0.0:1769074777.222211:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.222269:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.222436:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891648 00000100:00100000:2.0:1769074777.222447:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891648 00002000:00100000:2.0:1769074777.222499:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54385b480 x1855007616891648/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 504/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.222512:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b480 with x1855007616891648 ext(2097152->4194303) 00002000:00010000:2.0:1769074777.222518:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54385b480 00000100:00100000:2.0:1769074777.222548:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616891776 00000100:00100000:2.0:1769074777.222552:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616891776 00002000:00100000:2.0:1769074777.222568:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54385b800 x1855007616891776/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 504/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.222576:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b800 with x1855007616891776 ext(2097152->4194303) 00002000:00010000:2.0:1769074777.222887:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385b800 00000100:00100000:2.0:1769074777.222911:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8638 00000100:00100000:2.0:1769074777.222921:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616891648:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074777.223640:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8639 00000100:00100000:3.0:1769074777.223653:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616891776:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074777.263141:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616891776:12345-192.168.204.60@tcp:3:cat.0 Request processed in 39476us (40857us total) trans 0 rc 1880064/1880064 00000100:00100000:2.0:1769074777.263159:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8639 00002000:00100000:2.0:1769074777.263182:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54385b800 x1855007616891776/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 504/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1880064/1880064 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.263196:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b800 with x1855007616891776 ext(2097152->4194303) 00002000:00010000:2.0:1769074777.263203:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54385b800 00000100:00100000:2.0:1769074777.266672:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.266803:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892032 00000100:00100000:1.0:1769074777.266849:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892032 00000100:00100000:1.0:1769074777.266910:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13336 00000100:00100000:1.0:1769074777.266923:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+253:62979:x1855007616892032:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074777.266954:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074777.266983:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd54019 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfc6 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.267084:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267100:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267114:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267127:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267137:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267143:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267304:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5402e lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267317:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5402e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267325:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5402e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074777.267336:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074777.267436:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267443:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267449:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267456:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267461:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641650a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074777.267466:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267474:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267490:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074777.267493:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074777.267498:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267503:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650a00/0x7fe2323adfd54020 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267518:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267524:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267530:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267534:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641651a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074777.267537:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267542:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267549:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615c00) 00010000:00010000:1.0:1769074777.267551:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074777.267554:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267558:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651a00/0x7fe2323adfd54027 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074777.267598:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650800/0x7fe2323adfd54019 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfc6 expref: 256 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.267611:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5402e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfc6 expref: 255 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.267618:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074777.267620:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1769074777.267623:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:1.0:1769074777.267701:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:62979:x1855007616892032:12345-192.168.204.60@tcp:101:cat.0 Request processed in 779us (1039us total) trans 8589943979 rc 0/0 00000100:00100000:1.0:1769074777.267709:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13336 00000100:00100000:1.0:1769074777.267890:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616891648:12345-192.168.204.60@tcp:3:cat.0 Request processed in 44970us (45690us total) trans 0 rc 1380352/1380352 00000100:00100000:1.0:1769074777.267900:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8638 00002000:00100000:1.0:1769074777.267911:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54385b480 x1855007616891648/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 504/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1380352/1380352 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.267923:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385b480 with x1855007616891648 ext(2097152->4194303) 00002000:00010000:1.0:1769074777.267930:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54385b480 00000100:00100000:2.0:1769074777.280092:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.280147:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.280260:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892416 00000100:00100000:1.0:1769074777.280270:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892416 00000100:00100000:1.0:1769074777.280343:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1026 00000100:00100000:1.0:1769074777.280354:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:2406:x1855007616892416:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.280372:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.280398:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5402e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfc6 expref: 255 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.280425:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074777.280430:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.280496:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+254:2406:x1855007616892416:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 143us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.280503:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1026 00000100:00100000:0.0:1769074777.280638:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892288 00000100:00100000:0.0:1769074777.280644:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892288 00000100:00100000:0.0:1769074777.280684:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4684 00000100:00100000:0.0:1769074777.280692:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+254:62979:x1855007616892288:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:0.0:1769074777.280970:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:62979:x1855007616892288:12345-192.168.204.60@tcp:35:cat.0 Request processed in 277us (891us total) trans 8589943980 rc 0/0 00000100:00100000:0.0:1769074777.280978:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4684 00000100:00100000:1.0:1769074777.296588:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.296627:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.296659:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.296758:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892672 00000100:00100000:0.0:1769074777.296768:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892672 00000100:00100000:0.0:1769074777.296851:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892544 00000100:00100000:0.0:1769074777.296855:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892544 00000100:00100000:0.0:1769074777.296876:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892800 00000100:00100000:0.0:1769074777.296879:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892800 00000100:00100000:0.0:1769074777.296900:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14178 00000100:00100000:0.0:1769074777.296909:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616892672:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.296927:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.296945:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd5403c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfd4 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074777.297002:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625160832:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074777.297072:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ef5be40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074777.297091:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14179 00000100:00100000:0.0:1769074777.297097:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616892544:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.297104:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.297112:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfcd expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.297126:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.297128:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.297137:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.297154:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.297167:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:0.0:1769074777.297193:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616892544:12345-192.168.204.60@tcp:101:cat.0 Request processed in 96us (566us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.297204:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14179 00000100:00100000:0.0:1769074777.297223:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14180 00000100:00100000:0.0:1769074777.297228:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616892800:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.297245:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.297255:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5404a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfdb expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074777.297279:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625160960:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074777.297301:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074777.318360:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.318403:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:0.0:1769074777.318584:0:27844:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074774 -> 1769074776 00002000:00010000:0.0:1769074777.318589:0:27844:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074772 -> 1769074776 00002000:00010000:0.0:1769074777.318591:0:27844:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074774 -> 1769074776 00000100:00100000:0.0:1769074777.318611:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:27844:1855007625160960:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074777.318647:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5404a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfdb expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.318669:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052a00/0x7fe2323adfd5404a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfdb expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.318682:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:0.0:1769074777.318746:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616892800:12345-192.168.204.60@tcp:101:cat.0 Request processed in 21516us (22085us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074777.318756:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14180 00002000:00010000:0.0:1769074777.318843:0:28954:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074774 -> 1769074776 00002000:00010000:0.0:1769074777.318847:0:28954:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime: 1769074775 -> 1769074776 00000100:00100000:0.0:1769074777.318861:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625160832:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074777.318882:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd5403c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfd4 expref: 21 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.318896:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd5403c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfd4 expref: 21 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.318906:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:0.0:1769074777.318932:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616892672:12345-192.168.204.60@tcp:101:cat.0 Request processed in 22024us (22355us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074777.318939:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14178 00000100:00100000:0.0:1769074777.318961:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616892928 00000100:00100000:0.0:1769074777.318967:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616892928 00002000:00100000:0.0:1769074777.319006:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5436d7100 x1855007616892928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.319016:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7100 with x1855007616892928 ext(4194304->4808703) 00002000:00010000:0.0:1769074777.319021:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d7100 00000100:00100000:0.0:1769074777.319040:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8640 00000100:00100000:0.0:1769074777.319048:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616892928:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074777.321150:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616893184 00000100:00100000:3.0:1769074777.321166:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616893184 00000100:00100000:3.0:1769074777.321333:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13337 00000100:00100000:3.0:1769074777.321343:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:62980:x1855007616893184:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074777.321371:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074777.321394:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd54051 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfe2 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.321488:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321503:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321514:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321528:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321534:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321539:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321666:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ba00/0x7fe2323adfd54066 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321677:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ba00/0x7fe2323adfd54066 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321685:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ba00/0x7fe2323adfd54066 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:3.0:1769074777.321697:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:3.0:1769074777.321781:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321788:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321794:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321802:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321953:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773a600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074777.321957:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321965:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321977:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074777.321980:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074777.321985:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.321991:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a600/0x7fe2323adfd54058 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322006:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322012:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322021:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322026:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54773b000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074777.322028:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322033:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322039:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b6439ad800) 00010000:00010000:3.0:1769074777.322041:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074777.322044:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322049:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773b000/0x7fe2323adfd5405f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074777.322095:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773a400/0x7fe2323adfd54051 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfe2 expref: 258 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.322110:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ba00/0x7fe2323adfd54066 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfe2 expref: 257 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.322118:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074777.322120:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074777.322123:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54773ba00, rc 0) 00000100:00100000:3.0:1769074777.322286:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:62980:x1855007616893184:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 942us (3935us total) trans 8589943981 rc 0/0 00000100:00100000:3.0:1769074777.322296:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13337 00000100:00100000:0.0:1769074777.333538:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616892928:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14474us (15117us total) trans 0 rc 614400/614400 00000100:00100000:0.0:1769074777.333556:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8640 00002000:00100000:0.0:1769074777.333570:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5436d7100 x1855007616892928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.333586:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7100 with x1855007616892928 ext(4194304->4808703) 00002000:00010000:0.0:1769074777.333592:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d7100 00000100:00100000:2.0:1769074777.339044:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.339162:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616893568 00000100:00100000:1.0:1769074777.339173:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616893568 00000100:00100000:1.0:1769074777.339249:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4685 00000100:00100000:1.0:1769074777.339258:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c7850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:62980:x1855007616893568:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:1.0:1769074777.339523:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c7850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+258:62980:x1855007616893568:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 265us (488us total) trans 8589943982 rc 0/0 00000100:00100000:1.0:1769074777.339531:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4685 00000100:00100000:2.0:1769074777.341467:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.341549:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616893312 00000100:00100000:3.0:1769074777.341560:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616893312 00000100:00100000:3.0:1769074777.341637:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1027 00000100:00100000:3.0:1769074777.341648:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+258:2407:x1855007616893312:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074777.341665:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074777.341691:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54773ba00/0x7fe2323adfd54066 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfe2 expref: 258 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.341716:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074777.341722:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074777.341781:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:2407:x1855007616893312:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 135us (322us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074777.341789:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1027 00000100:00100000:2.0:1769074777.370925:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.371210:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616893696 00000100:00100000:0.0:1769074777.371222:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616893696 00002000:00100000:0.0:1769074777.371284:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5499d0a80 x1855007616893696/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.371296:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0a80 with x1855007616893696 ext(1048576->2097151) 00002000:00010000:0.0:1769074777.371302:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0a80 00000100:00100000:0.0:1769074777.371336:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8641 00000100:00100000:0.0:1769074777.371345:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616893696:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074777.391286:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616893696:12345-192.168.204.60@tcp:3:cat.0 Request processed in 19699us (20130us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074777.391301:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8641 00002000:00100000:0.0:1769074777.391315:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5499d0a80 x1855007616893696/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.391326:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0a80 with x1855007616893696 ext(1048576->2097151) 00002000:00010000:0.0:1769074777.391332:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0a80 00000100:00100000:2.0:1769074777.408912:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.408954:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.409071:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616893824 00000100:00100000:0.0:1769074777.409085:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616893824 00000100:00100000:0.0:1769074777.409145:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616893952 00000100:00100000:0.0:1769074777.409149:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616893952 00000100:00100000:0.0:1769074777.409203:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14181 00000100:00100000:0.0:1769074777.409214:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616893824:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.409246:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.409267:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54074 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bfe9 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074777.409322:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625161088:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074777.409417:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.409884:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14182 00000100:00100000:2.0:1769074777.409895:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616893952:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074777.409905:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.409916:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd5407b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bff0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074777.409946:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625161216:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074777.409981:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074777.413774:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625161216:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074777.413830:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd5407b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff0 expref: 16 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.413852:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd5407b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff0 expref: 16 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.413863:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:3.0:1769074777.413964:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616893952:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4070us (5007us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074777.413972:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14182 00000100:00100000:0.0:1769074777.414067:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625161088:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074777.414085:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54074 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfe9 expref: 21 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.414099:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54074 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfe9 expref: 21 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.414107:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:0.0:1769074777.414132:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616893824:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4921us (5233us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074777.414138:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14181 00000100:00100000:0.0:1769074777.420113:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.420257:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894080 00000100:00100000:2.0:1769074777.420267:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894080 00000100:00100000:2.0:1769074777.420328:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14183 00000100:00100000:2.0:1769074777.420337:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62979:x1855007616894080:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074777.420356:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.420375:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bff7 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.420407:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.420417:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.420420:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.420424:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.420465:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.420478:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 8 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.420492:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625161344:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.420569:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074777.422296:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625161344:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074777.422332:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.422350:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:0.0:1769074777.422406:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62979:x1855007616894080:12345-192.168.204.60@tcp:101:cat.0 Request processed in 2069us (2306us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.422418:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14183 00000100:00100000:1.0:1769074777.423396:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.423481:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894208 00000100:00100000:0.0:1769074777.423490:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894208 00002000:00100000:0.0:1769074777.423535:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3862527) req@ffff89b5436d6680 x1855007616894208/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.423546:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6680 with x1855007616894208 ext(3145728->3862527) 00010000:00010000:0.0:1769074777.423555:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 9 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074777.423569:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 9 pid: 8396 timeout: 1865 lvb_type: 0 00002000:00010000:0.0:1769074777.423576:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5436d6680 00000100:00100000:0.0:1769074777.423607:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 804 00000100:00100000:0.0:1769074777.423616:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2407:x1855007616894208:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:2.0:1769074777.432921:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.433030:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894336 00000100:00100000:0.0:1769074777.433040:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894336 00000100:00100000:0.0:1769074777.433113:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14184 00000100:00100000:0.0:1769074777.433122:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+11:62982:x1855007616894336:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074777.433139:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.433167:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879bffe expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.433196:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074777.433203:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.433209:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:0.0:1769074777.433213:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2269184->2695167] (req 2269184->2695167) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.433260:0:28954:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.433272:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2269184->2695167] (req 2269184->2695167) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 12 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.433284:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:0.0:1769074777.433346:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+12:62982:x1855007616894336:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 225us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.433353:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14184 00000100:00100000:1.0:1769074777.455619:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616894208:12345-192.168.204.60@tcp:4:cat.0 Request processed in 31990us (32214us total) trans 17179874527 rc 0/0 00000100:00100000:1.0:1769074777.455637:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 804 00002000:00100000:1.0:1769074777.455650:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->3862527) req@ffff89b5436d6680 x1855007616894208/t17179874527(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/448 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.455662:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6680 with x1855007616894208 ext(3145728->3862527) 00010000:00010000:1.0:1769074777.455671:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.455689:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00002000:00010000:1.0:1769074777.455697:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5436d6680 00000100:00100000:3.0:1769074777.467596:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.473902:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894464 00000100:00100000:1.0:1769074777.473918:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894464 00010000:00010000:1.0:1769074777.473969:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:1.0:1769074777.474085:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7092 00000100:00100000:1.0:1769074777.474096:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616894464:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.474112:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.474134:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.474148:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.474171:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d200/0x7fe2323adfd54012 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->3862527) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfbf expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.474186:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.474188:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.474195:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.474208:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.474282:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.474295:0:33102:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 20 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:1.0:1769074777.474308:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625161600:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074777.474411:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074777.485766:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625161600:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074777.485796:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074777.485948:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2407:x1855007616894464:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 11851us (18360us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.485959:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7092 00000100:00100000:2.0:1769074777.490267:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.490379:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894592 00000100:00100000:1.0:1769074777.490401:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894592 00002000:00100000:1.0:1769074777.490463:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5499d2680 x1855007616894592/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.490635:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2680 with x1855007616894592 ext(2097152->3145727) 00010000:00010000:1.0:1769074777.490648:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.490666:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00002000:00010000:1.0:1769074777.490678:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d2680 00000100:00100000:1.0:1769074777.490719:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 805 00000100:00100000:1.0:1769074777.490729:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616894592:12345-192.168.204.60@tcp:3:cat.0 00010000:00010000:0.0:1769074777.504682:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074777.521011:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616894592:12345-192.168.204.60@tcp:3:cat.0 Request processed in 30272us (30742us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074777.521027:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 805 00002000:00100000:1.0:1769074777.521045:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->3145727) req@ffff89b5499d2680 x1855007616894592/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.521057:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2680 with x1855007616894592 ext(2097152->3145727) 00010000:00010000:1.0:1769074777.521069:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.521086:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00002000:00010000:1.0:1769074777.521096:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d2680 00000100:00100000:0.0:1769074777.534603:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.534717:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894720 00000100:00100000:2.0:1769074777.534729:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894720 00000100:00100000:2.0:1769074777.534801:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14185 00000100:00100000:2.0:1769074777.534847:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:62979:x1855007616894720:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074777.534869:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.534897:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c005 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.534929:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 11 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.534937:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.534939:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.534944:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.534982:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 11 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.534998:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 11 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.535013:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625161728:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.535107:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.540672:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.540999:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894848 00000100:00100000:0.0:1769074777.541011:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894848 00010000:00010000:0.0:1769074777.541054:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074777.541101:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7093 00000100:00100000:0.0:1769074777.541112:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616894848:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074777.541131:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074777.541154:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074777.541168:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074777.541196:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa400/0x7fe2323adfd54082 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bff7 expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074777.541214:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.541216:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.541225:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.541246:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.541287:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074777.541301:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625161856:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074777.541535:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074777.543566:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.543690:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616894976 00000100:00100000:3.0:1769074777.543700:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616894976 00002000:00100000:3.0:1769074777.543750:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2928640->3145727) req@ffff89b67fa22680 x1855007616894976/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.543762:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22680 with x1855007616894976 ext(2928640->3145727) 00010000:00010000:3.0:1769074777.543772:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074777.543786:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 12 pid: 8396 timeout: 1865 lvb_type: 0 00002000:00010000:3.0:1769074777.543794:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fa22680 00000100:00100000:3.0:1769074777.543849:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 806 00000100:00100000:3.0:1769074777.543858:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616894976:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:2.0:1769074777.544304:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625161728:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.544332:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.544347:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:2.0:1769074777.544389:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:62979:x1855007616894720:12345-192.168.204.60@tcp:101:cat.0 Request processed in 9535us (9786us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.544397:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14185 00000100:00100000:0.0:1769074777.546544:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625161856:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074777.546583:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074777.546648:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616894848:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5536us (5984us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.546659:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7093 00000100:00100000:2.0:1769074777.559670:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.559798:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895104 00000100:00100000:0.0:1769074777.559854:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895104 00002000:00100000:0.0:1769074777.559928:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2269184->2273279) req@ffff89b5499d0700 x1855007616895104/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.559940:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0700 with x1855007616895104 ext(2269184->2273279) 00002000:00010000:0.0:1769074777.559947:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0700 00000100:00100000:0.0:1769074777.559993:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8642 00000100:00100000:0.0:1769074777.560004:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616895104:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074777.560706:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616895104:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 699us (1043us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074777.560716:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8642 00002000:00100000:0.0:1769074777.560724:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2269184->2273279) req@ffff89b5499d0700 x1855007616895104/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.560734:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0700 with x1855007616895104 ext(2269184->2273279) 00002000:00010000:0.0:1769074777.560738:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0700 00000100:00100000:2.0:1769074777.580977:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616894976:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 37107us (37419us total) trans 21474841206 rc 0/0 00000100:00100000:2.0:1769074777.580991:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 806 00002000:00100000:2.0:1769074777.581004:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (2928640->3145727) req@ffff89b67fa22680 x1855007616894976/t21474841206(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/448 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.581016:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22680 with x1855007616894976 ext(2928640->3145727) 00010000:00010000:2.0:1769074777.581025:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.581927:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00002000:00010000:2.0:1769074777.581940:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67fa22680 00000100:00100000:3.0:1769074777.583978:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.584281:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895232 00000100:00100000:2.0:1769074777.584293:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895232 00002000:00100000:2.0:1769074777.584341:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2691072->2695167) req@ffff89b54ce9d500 x1855007616895232/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.584353:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9d500 with x1855007616895232 ext(2691072->2695167) 00002000:00010000:2.0:1769074777.584359:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9d500 00000100:00100000:2.0:1769074777.584399:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8643 00000100:00100000:2.0:1769074777.584409:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616895232:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:2.0:1769074777.584584:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616895232:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 176us (615us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074777.584592:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8643 00002000:00100000:2.0:1769074777.584599:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2691072->2695167) req@ffff89b54ce9d500 x1855007616895232/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 4096/4096 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.584606:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9d500 with x1855007616895232 ext(2691072->2695167) 00002000:00010000:2.0:1769074777.584610:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54ce9d500 00000100:00100000:0.0:1769074777.603021:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.604176:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895360 00000100:00100000:2.0:1769074777.604190:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895360 00010000:00010000:2.0:1769074777.604249:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.604299:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7094 00000100:00100000:2.0:1769074777.604309:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2408:x1855007616895360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074777.604322:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074777.604344:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.604357:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.604378:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54004 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2928640->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb1 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.604404:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.604406:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.604415:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.604426:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.604465:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074777.604478:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625161984:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074777.604569:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f30c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074777.626745:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.626783:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.626938:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625161984:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074777.626967:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074777.627030:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616895360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 22722us (24018us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.627043:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7094 00000100:00100000:1.0:1769074777.627177:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895488 00000100:00100000:1.0:1769074777.627186:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895488 00000100:00100000:1.0:1769074777.627244:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14186 00000100:00100000:1.0:1769074777.627253:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616895488:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074777.627271:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074777.627289:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c00c expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.627316:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.627318:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.627328:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 14 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074777.627345:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 14 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074777.627358:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b437600, rc 0) 00000100:00100000:1.0:1769074777.627381:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616895488:12345-192.168.204.60@tcp:101:cat.0 Request processed in 129us (649us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.627387:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14186 00000100:00100000:3.0:1769074777.630153:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895616 00000100:00100000:3.0:1769074777.630173:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895616 00002000:00100000:3.0:1769074777.630241:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b67fa21880 x1855007616895616/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.630256:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa21880 with x1855007616895616 ext(2097152->3145727) 00002000:00010000:3.0:1769074777.630261:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fa21880 00000100:00100000:3.0:1769074777.630302:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8644 00000100:00100000:3.0:1769074777.630312:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2406:x1855007616895616:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074777.651438:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2406:x1855007616895616:12345-192.168.204.60@tcp:3:cat.0 Request processed in 21114us (24641us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074777.651453:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8644 00002000:00100000:3.0:1769074777.651466:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->3145727) req@ffff89b67fa21880 x1855007616895616/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.651476:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa21880 with x1855007616895616 ext(2097152->3145727) 00002000:00010000:3.0:1769074777.651482:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67fa21880 00000100:00100000:0.0:1769074777.661051:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.661158:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895744 00000100:00100000:2.0:1769074777.661169:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895744 00000100:00100000:2.0:1769074777.661246:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14187 00000100:00100000:2.0:1769074777.661257:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:62979:x1855007616895744:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074777.661274:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.661295:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c013 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.661325:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.661333:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.661335:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.661340:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.661374:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.661400:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 10 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.661414:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625162112:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074777.661499:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.669870:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625162112:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.669907:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.669928:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:2.0:1769074777.669998:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62979:x1855007616895744:12345-192.168.204.60@tcp:101:cat.0 Request processed in 8742us (8957us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.670007:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14187 00000100:00100000:0.0:1769074777.680693:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.680865:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616895872 00000100:00100000:3.0:1769074777.680875:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616895872 00002000:00100000:3.0:1769074777.680926:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2269184->2695167) req@ffff89b67fa20000 x1855007616895872/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.680937:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa20000 with x1855007616895872 ext(2269184->2695167) 00010000:00010000:3.0:1769074777.680945:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 11 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074777.680960:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 11 pid: 28954 timeout: 1865 lvb_type: 0 00002000:00010000:3.0:1769074777.680968:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67fa20000 00000100:00100000:3.0:1769074777.681004:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 807 00000100:00100000:3.0:1769074777.681014:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616895872:12345-192.168.204.60@tcp:4:multiop.0 00000100:00100000:3.0:1769074777.690305:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.691425:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896000 00000100:00100000:2.0:1769074777.691441:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896000 00000100:00100000:2.0:1769074777.693590:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14188 00000100:00100000:2.0:1769074777.693626:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62987:x1855007616896000:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074777.693655:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.693683:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c01a expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.693731:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.693733:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074777.693739:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.693751:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.693764:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:2.0:1769074777.693869:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62987:x1855007616896000:12345-192.168.204.60@tcp:101:cat.0 Request processed in 262us (3574us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.693877:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14188 00000100:00100000:3.0:1769074777.705719:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616895872:12345-192.168.204.60@tcp:4:multiop.0 Request processed in 24692us (25022us total) trans 17179874528 rc 0/0 00000100:00100000:3.0:1769074777.705735:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 807 00002000:00100000:3.0:1769074777.705748:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2269184->2695167) req@ffff89b67fa20000 x1855007616895872/t17179874528(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/448 e 0 to 0 dl 1769074788 ref 1 fl Complete:H/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.705759:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa20000 with x1855007616895872 ext(2269184->2695167) 00010000:00010000:3.0:1769074777.705769:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 14 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074777.705788:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 14 pid: 28954 timeout: 1865 lvb_type: 0 00002000:00010000:3.0:1769074777.705797:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67fa20000 00000100:00100000:0.0:1769074777.715282:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.716199:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896128 00000100:00100000:2.0:1769074777.716214:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896128 00010000:00010000:2.0:1769074777.716275:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 14 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.716323:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7095 00000100:00100000:2.0:1769074777.716333:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616896128:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074777.716350:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1769074777.716368:0:33102:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074776 -> 1769074777 00002000:00010000:2.0:1769074777.716371:0:33102:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074776 -> 1769074777 00010000:00010000:2.0:1769074777.716378:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 14 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.716403:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 14 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.716427:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd54089 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2269184->2695167) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bffe expref: 14 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.716443:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.716445:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.716453:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.716463:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.716471:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.716473:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.716477:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.716484:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.716519:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 13 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074777.716533:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625162240:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074777.717412:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074777.717425:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625162368:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074777.717457:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074777.728536:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625162240:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074777.728555:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625162368:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074777.728591:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074777.728660:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616896128:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 12327us (13386us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.728672:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7095 00000100:00100000:0.0:1769074777.755477:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.755780:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896256 00000100:00100000:3.0:1769074777.755794:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896256 00002000:00100000:3.0:1769074777.755882:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67fa22300 x1855007616896256/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074777.755894:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22300 with x1855007616896256 ext(3145728->4194303) 00002000:00010000:3.0:1769074777.755900:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa22300 00000100:00100000:3.0:1769074777.755936:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8645 00000100:00100000:3.0:1769074777.755945:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616896256:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074777.764175:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.764275:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896384 00000100:00100000:2.0:1769074777.764286:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896384 00002000:00100000:2.0:1769074777.764332:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54ce9c700 x1855007616896384/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.764345:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9c700 with x1855007616896384 ext(3145728->4194303) 00002000:00010000:2.0:1769074777.764351:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9c700 00000100:00100000:2.0:1769074777.764380:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8646 00000100:00100000:2.0:1769074777.764388:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616896384:12345-192.168.204.60@tcp:3:multiop.0 00000100:00100000:0.0:1769074777.775748:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.777059:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896512 00000100:00100000:2.0:1769074777.777078:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896512 00002000:00100000:2.0:1769074777.777142:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4808703) req@ffff89b67fa21180 x1855007616896512/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.777155:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa21180 with x1855007616896512 ext(2097152->4808703) 00002000:00010000:2.0:1769074777.777163:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa21180 00000100:00100000:2.0:1769074777.777195:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8647 00000100:00100000:2.0:1769074777.777205:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616896512:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074777.780914:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.781011:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896640 00000100:00100000:0.0:1769074777.781024:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896640 00002000:00100000:0.0:1769074777.781079:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5436d6300 x1855007616896640/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074777.781090:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6300 with x1855007616896640 ext(2097152->4194303) 00002000:00010000:0.0:1769074777.781097:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d6300 00000100:00100000:0.0:1769074777.781132:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8648 00000100:00100000:0.0:1769074777.781143:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616896640:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074777.787142:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616896256:12345-192.168.204.60@tcp:3:cat.0 Request processed in 31183us (31663us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074777.787164:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8645 00002000:00100000:2.0:1769074777.787181:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b67fa22300 x1855007616896256/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.787193:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22300 with x1855007616896256 ext(3145728->4194303) 00002000:00010000:2.0:1769074777.787201:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa22300 00000100:00100000:0.0:1769074777.814930:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.815068:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896768 00000100:00100000:2.0:1769074777.815082:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896768 00000100:00100000:2.0:1769074777.815151:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14189 00000100:00100000:2.0:1769074777.815160:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62980:x1855007616896768:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:2.0:1769074777.815179:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074777.815200:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c021 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.815241:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.815254:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.815261:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.815263:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074777.815267:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.815304:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.815318:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.815332:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625162496:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074777.815601:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074777.815612:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.815621:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625162624:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074777.816586:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616896384:12345-192.168.204.60@tcp:3:multiop.0 Request processed in 52193us (52416us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074777.816596:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8646 00002000:00100000:1.0:1769074777.816607:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x2bc:0x0] (3145728->4194303) req@ffff89b54ce9c700 x1855007616896384/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'multiop.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.816617:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54ce9c700 with x1855007616896384 ext(3145728->4194303) 00002000:00010000:1.0:1769074777.816623:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54ce9c700 00000100:00100000:2.0:1769074777.833249:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.835516:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616896512:12345-192.168.204.60@tcp:3:cat.0 Request processed in 58301us (59770us total) trans 0 rc 2711552/2711552 00000100:00100000:2.0:1769074777.835534:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8647 00002000:00100000:2.0:1769074777.835549:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4808703) req@ffff89b67fa21180 x1855007616896512/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 2711552/2711552 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.835561:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa21180 with x1855007616896512 ext(2097152->4808703) 00010000:00010000:2.0:1769074777.835572:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 13 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.835588:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 13 pid: 8396 timeout: 1865 lvb_type: 0 00002000:00010000:2.0:1769074777.835597:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67fa21180 00000100:00100000:3.0:1769074777.836326:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625162496:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074777.843638:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625162624:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074777.843685:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.843709:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fae00, rc 0) 00000100:00100000:3.0:1769074777.843794:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62980:x1855007616896768:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 28633us (28877us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074777.843845:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14189 00000100:00100000:1.0:1769074777.845507:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.845686:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616896896 00000100:00100000:3.0:1769074777.845696:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616896896 00010000:00010000:3.0:1769074777.845734:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:3.0:1769074777.845779:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7096 00000100:00100000:3.0:1769074777.845788:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616896896:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074777.845803:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074777.845844:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074777.845857:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074777.845879:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb800/0x7fe2323adfd5409e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c013 expref: 22 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074777.845898:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074777.845943:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616896896:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 156us (447us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074777.845950:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7096 00000100:00100000:1.0:1769074777.847882:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616896640:12345-192.168.204.60@tcp:3:cat.0 Request processed in 66726us (66964us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074777.847898:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8648 00002000:00100000:1.0:1769074777.847915:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b5436d6300 x1855007616896640/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.847926:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6300 with x1855007616896640 ext(2097152->4194303) 00002000:00010000:1.0:1769074777.847934:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d6300 00000100:00100000:3.0:1769074777.848381:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.849174:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897024 00000100:00100000:0.0:1769074777.849184:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897024 00000100:00100000:0.0:1769074777.849243:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14190 00000100:00100000:0.0:1769074777.849252:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616897024:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.849269:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.849288:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540b3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c028 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.849311:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.849323:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540b3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c028 expref: 22 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.849340:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540b3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c028 expref: 22 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.849352:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5200, rc 0) 00000100:00100000:0.0:1769074777.849407:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616897024:12345-192.168.204.60@tcp:101:cat.0 Request processed in 155us (1027us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074777.849415:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14190 00000100:00100000:3.0:1769074777.854927:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.855077:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897152 00000100:00100000:1.0:1769074777.855090:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897152 00000100:00100000:1.0:1769074777.855143:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14191 00000100:00100000:1.0:1769074777.855153:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:62979:x1855007616897152:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074777.855175:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074777.855195:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c02f expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.855231:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.855234:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.855244:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.855261:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 22 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.855275:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876c00, rc 0) 00000100:00100000:1.0:1769074777.855339:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:62979:x1855007616897152:12345-192.168.204.60@tcp:101:cat.0 Request processed in 188us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.855347:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14191 00000100:00100000:2.0:1769074777.862715:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.862847:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897280 00000100:00100000:0.0:1769074777.862860:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897280 00000100:00100000:0.0:1769074777.862914:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14192 00000100:00100000:0.0:1769074777.862924:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62987:x1855007616897280:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.862946:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.862969:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c036 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.863000:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.863003:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074777.863013:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.863035:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.863061:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b6725b5200, rc 0) 00000100:00100000:0.0:1769074777.863138:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+14:62987:x1855007616897280:12345-192.168.204.60@tcp:101:cat.0 Request processed in 214us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.863145:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14192 00000100:00100000:2.0:1769074777.872265:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.872373:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897408 00000100:00100000:1.0:1769074777.872384:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897408 00002000:00100000:1.0:1769074777.872438:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5499d3100 x1855007616897408/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.872451:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d3100 with x1855007616897408 ext(4194304->4808703) 00002000:00010000:1.0:1769074777.872456:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d3100 00000100:00100000:1.0:1769074777.872498:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8649 00000100:00100000:1.0:1769074777.872508:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616897408:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074777.884644:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616897408:12345-192.168.204.60@tcp:3:cat.0 Request processed in 12121us (12374us total) trans 0 rc 614400/614400 00000100:00100000:1.0:1769074777.884659:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8649 00002000:00100000:1.0:1769074777.884672:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5499d3100 x1855007616897408/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.884684:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d3100 with x1855007616897408 ext(4194304->4808703) 00002000:00010000:1.0:1769074777.884690:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d3100 00000100:00100000:2.0:1769074777.894685:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.897869:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897536 00000100:00100000:1.0:1769074777.897898:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897536 00010000:00010000:1.0:1769074777.897953:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:1.0:1769074777.898006:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7097 00000100:00100000:1.0:1769074777.898018:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2409:x1855007616897536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074777.898034:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074777.898234:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.898251:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.898278:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd540a5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c01a expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:1.0:1769074777.898295:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.898298:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074777.898306:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.898316:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074777.898358:0:6499:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074777.898372:0:6499:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625162752:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074777.898477:0:6499:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074777.898754:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.899016:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897664 00000100:00100000:1.0:1769074777.899023:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897664 00002000:00100000:1.0:1769074777.899174:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5499d2300 x1855007616897664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.899183:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2300 with x1855007616897664 ext(3145728->4194303) 00002000:00010000:1.0:1769074777.899189:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d2300 00000100:00100000:1.0:1769074777.899217:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8650 00000100:00100000:1.0:1769074777.899235:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2406:x1855007616897664:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074777.926396:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074777.929142:0:6499:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6499:1855007625162752:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074777.929175:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074777.929276:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616897536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 31259us (34600us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074777.929288:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7097 00000100:00100000:2.0:1769074777.930636:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897792 00000100:00100000:2.0:1769074777.930655:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897792 00002000:00100000:2.0:1769074777.930721:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3088384->3092479) req@ffff89b67fa22a00 x1855007616897792/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/0 e 0 to 0 dl 1769074788 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.930734:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22a00 with x1855007616897792 ext(3088384->3092479) 00002000:00010000:2.0:1769074777.930741:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa22a00 00000100:00100000:2.0:1769074777.930775:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8651 00000100:00100000:2.0:1769074777.930786:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616897792:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074777.932331:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616897792:12345-192.168.204.60@tcp:3:cat.0 Request processed in 1533us (5932us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1769074777.932345:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8651 00002000:00100000:2.0:1769074777.932357:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3088384->3092479) req@ffff89b67fa22a00 x1855007616897792/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:313/0 lens 488/4536 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074777.932367:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67fa22a00 with x1855007616897792 ext(3088384->3092479) 00002000:00010000:2.0:1769074777.932372:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67fa22a00 00000100:00100000:1.0:1769074777.954503:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.956360:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616897920 00000100:00100000:3.0:1769074777.956386:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616897920 00000100:00100000:3.0:1769074777.956529:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14193 00000100:00100000:3.0:1769074777.956539:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616897920:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074777.956560:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074777.956586:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd540c8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c03d expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.956621:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd540c8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c03d expref: 23 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.956640:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866e00/0x7fe2323adfd540c8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c03d expref: 23 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.956653:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:3.0:1769074777.956725:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616897920:12345-192.168.204.60@tcp:101:cat.0 Request processed in 188us (2234us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074777.956732:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14193 00000100:00100000:1.0:1769074777.963120:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616897664:12345-192.168.204.60@tcp:3:cat.0 Request processed in 63872us (64353us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074777.963136:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8650 00002000:00100000:1.0:1769074777.963150:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3145728->4194303) req@ffff89b5499d2300 x1855007616897664/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:313/0 lens 488/440 e 0 to 0 dl 1769074788 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074777.963160:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d2300 with x1855007616897664 ext(3145728->4194303) 00002000:00010000:1.0:1769074777.963166:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d2300 00000100:00100000:0.0:1769074777.968022:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.968074:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898048 00000100:00100000:3.0:1769074777.968083:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898048 00000100:00100000:0.0:1769074777.968126:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.974047:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898176 00000100:00100000:3.0:1769074777.974066:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898176 00000100:00100000:3.0:1769074777.974252:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14194 00000100:00100000:3.0:1769074777.974263:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:62980:x1855007616898048:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074777.974284:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074777.974305:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c044 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.974336:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.974352:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.974359:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074777.974361:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074777.974365:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->2998271] (req 2097152->2998271) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.974418:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.974433:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 13 pid: 28954 timeout: 1865 lvb_type: 1 00000100:00100000:3.0:1769074777.974445:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625162880:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074777.974549:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.974558:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074777.974596:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074777.974720:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898432 00000100:00100000:0.0:1769074777.974725:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898432 00000100:00100000:0.0:1769074777.974753:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14195 00000100:00100000:0.0:1769074777.974760:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616898176:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074777.974769:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.974780:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd540d6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c04b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.974802:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd540d6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c04b expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.974836:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd540d6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c04b expref: 23 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074777.974848:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45800, rc 0) 00000100:00100000:0.0:1769074777.974887:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616898176:12345-192.168.204.60@tcp:101:cat.0 Request processed in 128us (6760us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074777.974894:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14195 00000100:00100000:0.0:1769074777.974913:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14196 00000100:00100000:0.0:1769074777.974919:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62989:x1855007616898432:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074777.974930:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074777.974938:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c059 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.974953:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.974962:0:27844:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.974981:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:0.0:1769074777.974982:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:0.0:1769074777.974986:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3829760->4018175] (req 3829760->4018175) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.975007:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 14 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.975015:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074777.975024:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625163136:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074777.975060:0:27844:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 21 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.975069:0:27844:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 21 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074777.975085:0:27844:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625163264:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074777.975108:0:27844:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074777.976835:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625163008:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074777.976956:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074777.979078:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074777.979119:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898304 00000100:00100000:3.0:1769074777.979132:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898304 00000100:00100000:3.0:1769074777.979189:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14197 00000100:00100000:3.0:1769074777.979199:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616898304:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074777.979232:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074777.979253:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd540e4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c052 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074777.979290:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd540e4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c052 expref: 16 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.979309:0:8399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd540e4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c052 expref: 16 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074777.979321:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:3.0:1769074777.979370:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2409:x1855007616898304:12345-192.168.204.60@tcp:101:cat.0 Request processed in 172us (304us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074777.980155:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14197 00000100:00100000:0.0:1769074777.980246:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625162880:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074777.985722:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074777.986331:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898560 00000100:00100000:2.0:1769074777.986343:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898560 00010000:00010000:2.0:1769074777.986397:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 15 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:2.0:1769074777.986441:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7098 00000100:00100000:2.0:1769074777.986453:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616898560:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074777.986470:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074777.986492:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 15 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.986505:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 15 pid: 8396 timeout: 1865 lvb_type: 0 00010000:00010000:2.0:1769074777.986533:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074777.986783:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2406:x1855007616898560:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 332us (1070us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074777.986792:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7098 00010000:00010000:0.0:1769074777.986859:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5200/0x7fe2323adfd540c1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c036 expref: 14 pid: 8396 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074777.986875:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625163136:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074777.986889:0:27844:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:27844:1855007625163264:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074777.986924:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->4018175] (req 3829760->4018175) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074777.986938:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45200, rc 0) 00000100:00100000:0.0:1769074777.986970:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:543044d2-7a16-4e58-9b2c-28bb782e5779+13:62989:x1855007616898432:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 12050us (12374us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074777.986976:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14196 00000100:00100000:0.0:1769074778.000719:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.000777:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.000999:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898816 00000100:00100000:3.0:1769074778.001011:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898816 00010000:00010000:3.0:1769074778.001068:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 22 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:3.0:1769074778.001110:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898688 00000100:00100000:3.0:1769074778.001114:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898688 00010000:00010000:3.0:1769074778.001128:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 14 pid: 28954 timeout: 1865 lvb_type: 1 00000100:00100000:3.0:1769074778.001168:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7099 00000100:00100000:3.0:1769074778.001178:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616898816:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.001196:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.001229:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 22 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.001243:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 22 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.001268:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876c00/0x7fe2323adfd540ba lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3145728->18446744073709551615] (req 4194304->4808703) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c02f expref: 22 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.001284:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.001286:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.001294:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.001304:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.001344:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074778.001358:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625163392:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074778.001452:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.001887:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7100 00000100:00100000:2.0:1769074778.001899:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616898688:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.001910:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074778.001926:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 14 pid: 28954 timeout: 1865 lvb_type: 1 00010000:00010000:2.0:1769074778.001937:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 14 pid: 28954 timeout: 1865 lvb_type: 1 00010000:00010000:2.0:1769074778.001953:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b437600/0x7fe2323adfd54097 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c00c expref: 14 pid: 28954 timeout: 1865 lvb_type: 1 00010000:00010000:2.0:1769074778.001965:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074778.002009:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616898688:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 113us (1232us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.002017:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7100 00000100:00100000:3.0:1769074778.010562:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625163008:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.010619:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->2998271] (req 2097152->2998271) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 17 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.010645:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:3.0:1769074778.010728:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:62980:x1855007616898048:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 36466us (42714us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.010740:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14194 00000100:00100000:0.0:1769074778.015774:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:33102:1855007625163392:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074778.015803:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.015912:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616898816:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 14734us (15206us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.015924:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7099 00000100:00100000:0.0:1769074778.020009:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.020071:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.020171:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899072 00000100:00100000:1.0:1769074778.020185:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899072 00002000:00100000:1.0:1769074778.020257:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3829760->3833855) req@ffff89b543858000 x1855007616899072/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074778.020268:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858000 with x1855007616899072 ext(3829760->3833855) 00002000:00010000:1.0:1769074778.020275:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543858000 00000100:00100000:1.0:1769074778.020311:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8652 00000100:00100000:1.0:1769074778.020320:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616899072:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074778.020531:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616899072:12345-192.168.204.60@tcp:3:cat.0 Request processed in 211us (459us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074778.020539:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8652 00002000:00100000:1.0:1769074778.020545:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3829760->3833855) req@ffff89b543858000 x1855007616899072/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/4536 e 0 to 0 dl 1769074789 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074778.020554:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858000 with x1855007616899072 ext(3829760->3833855) 00002000:00010000:1.0:1769074778.020557:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543858000 00000100:00100000:3.0:1769074778.022327:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616898944 00000100:00100000:3.0:1769074778.022350:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616898944 00010000:00010000:3.0:1769074778.022448:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00100000:3.0:1769074778.022498:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7101 00000100:00100000:3.0:1769074778.022508:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e667c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616898944:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.022522:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.022542:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.022556:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 1865 lvb_type: 0 00000100:00080000:2.0:1769074778.022767:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1769074778.022845:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54090 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c005 expref: 17 pid: 28954 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.022868:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.022870:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.022878:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.022888:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.022928:0:33102:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074778.022942:0:33102:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625163520:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074778.023081:0:33102:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.033109:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.033179:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.033339:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899200 00000100:00100000:3.0:1769074778.033351:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899200 00002000:00100000:3.0:1769074778.033415:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4014080->4018175) req@ffff89b5499d0e00 x1855007616899200/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074778.033426:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0e00 with x1855007616899200 ext(4014080->4018175) 00002000:00010000:3.0:1769074778.033433:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0e00 00000100:00100000:3.0:1769074778.033470:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899328 00000100:00100000:3.0:1769074778.033474:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899328 00002000:00100000:3.0:1769074778.033494:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5499d3800 x1855007616899328/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074778.033503:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d3800 with x1855007616899328 ext(4194304->4808703) 00002000:00010000:3.0:1769074778.033506:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d3800 00000100:00100000:3.0:1769074778.033524:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8653 00000100:00100000:3.0:1769074778.033533:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616899200:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074778.033710:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+14:2407:x1855007616899200:12345-192.168.204.60@tcp:3:cat.0 Request processed in 177us (613us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074778.033718:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8653 00002000:00100000:3.0:1769074778.033725:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4014080->4018175) req@ffff89b5499d0e00 x1855007616899200/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/4536 e 0 to 0 dl 1769074789 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074778.033733:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0e00 with x1855007616899200 ext(4014080->4018175) 00002000:00010000:3.0:1769074778.033736:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0e00 00000100:00100000:3.0:1769074778.033754:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8654 00000100:00100000:3.0:1769074778.033762:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899328:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074778.042358:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899328:12345-192.168.204.60@tcp:3:cat.0 Request processed in 8585us (9167us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074778.042384:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8654 00002000:00100000:3.0:1769074778.042400:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5499d3800 x1855007616899328/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/440 e 0 to 0 dl 1769074789 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074778.042412:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d3800 with x1855007616899328 ext(4194304->4808703) 00002000:00010000:3.0:1769074778.042419:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d3800 00000100:00100000:3.0:1769074778.048372:0:33102:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:33102:1855007625163520:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074778.048410:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.048479:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e667c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616898944:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 25971us (28486us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.048490:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7101 00000100:00100000:2.0:1769074778.053309:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.053527:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.053703:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899456 00000100:00100000:0.0:1769074778.053716:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899456 00000100:00100000:0.0:1769074778.053835:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899584 00000100:00100000:0.0:1769074778.053839:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899584 00000100:00100000:1.0:1769074778.053965:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14198 00000100:00100000:1.0:1769074778.053978:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899456:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074778.053999:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.054017:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd540eb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c060 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074778.054094:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625163648:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074778.054182:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.057349:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14199 00000100:00100000:2.0:1769074778.057382:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899584:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074778.057400:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.057418:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd540f2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c067 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074778.057478:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625163776:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074778.057557:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00002000:00010000:1.0:1769074778.058546:0:8396:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime: 1769074776 -> 1769074777 00000100:00100000:1.0:1769074778.058625:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625163648:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074778.058652:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd540eb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c060 expref: 14 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.058668:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd540eb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c060 expref: 14 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.058677:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:1.0:1769074778.058744:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899456:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4770us (5450us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074778.058751:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14198 00000100:00100000:0.0:1769074778.062588:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00002000:00010000:2.0:1769074778.062788:0:28955:0:(ofd_lvb.c:264:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime: 1769074776 -> 1769074777 00002000:00010000:2.0:1769074778.062791:0:28955:0:(ofd_lvb.c:270:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime: 1769074776 -> 1769074777 00002000:00010000:2.0:1769074778.062793:0:28955:0:(ofd_lvb.c:276:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime: 1769074776 -> 1769074777 00000100:00100000:2.0:1769074778.062843:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625163776:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074778.062870:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd540f2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c067 expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.062887:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd540f2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c067 expref: 14 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.062895:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:2.0:1769074778.062940:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899584:12345-192.168.204.60@tcp:101:cat.0 Request processed in 5581us (9414us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074778.062947:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14199 00000100:00100000:3.0:1769074778.063341:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899712 00000100:00100000:3.0:1769074778.063348:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899712 00002000:00100000:0.0:1769074778.063497:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2994176->2998271) req@ffff89b543859180 x1855007616899712/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074778.063504:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859180 with x1855007616899712 ext(2994176->2998271) 00002000:00010000:0.0:1769074778.063508:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543859180 00000100:00100000:0.0:1769074778.063529:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8655 00000100:00100000:0.0:1769074778.063536:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616899712:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074778.063656:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616899712:12345-192.168.204.60@tcp:3:cat.0 Request processed in 120us (1076us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074778.063663:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8655 00002000:00100000:0.0:1769074778.063668:0:28448:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_015: refresh rw locks for [0x280000401:0x2bc:0x0] (2994176->2998271) req@ffff89b543859180 x1855007616899712/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:314/0 lens 488/4536 e 0 to 0 dl 1769074789 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074778.063677:0:28448:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543859180 with x1855007616899712 ext(2994176->2998271) 00002000:00010000:0.0:1769074778.063680:0:28448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b543859180 00000100:00100000:0.0:1769074778.068728:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.068851:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.071792:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899840 00000100:00100000:2.0:1769074778.071903:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899840 00000100:00100000:2.0:1769074778.075332:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616899968 00000100:00100000:2.0:1769074778.075348:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616899968 00000100:00100000:1.0:1769074778.075594:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14200 00000100:00100000:1.0:1769074778.075608:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899840:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074778.075630:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.075649:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd540f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c06e expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074778.075711:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625163904:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074778.075821:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b544abb780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074778.077015:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14201 00000100:00100000:0.0:1769074778.077835:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899968:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074778.077856:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074778.077875:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54100 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c075 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074778.077928:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625164032:192.168.204.60@tcp:106: 00000100:00100000:0.0:1769074778.078024:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.081508:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625164032:192.168.204.60@tcp:106: 00010000:00010000:2.0:1769074778.081547:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54100 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c075 expref: 14 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.081584:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54100 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c075 expref: 14 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.081602:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44000, rc 0) 00000100:00100000:2.0:1769074778.081659:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899968:12345-192.168.204.60@tcp:101:cat.0 Request processed in 4633us (12807us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074778.081669:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14201 00000100:00100000:1.0:1769074778.082205:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625163904:192.168.204.60@tcp:106: 00010000:00010000:1.0:1769074778.082238:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd540f9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c06e expref: 14 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.082251:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd540f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c06e expref: 14 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.082261:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:1.0:1769074778.082292:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616899840:12345-192.168.204.60@tcp:101:cat.0 Request processed in 6688us (13575us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074778.082298:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14200 00000100:00100000:0.0:1769074778.110284:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.110398:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616900096 00000100:00100000:1.0:1769074778.110410:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616900096 00000100:00100000:1.0:1769074778.110498:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13338 00000100:00100000:1.0:1769074778.110510:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:62993:x1855007616900096:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.110535:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.110562:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54107 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c07c expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.110647:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110661:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110672:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110687:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110694:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110700:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110772:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.110784:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 256 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.110792:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.110799:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 5/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00000100:00080000:2.0:1769074778.111190:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1769074778.111985:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.112005:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 10 pid: 8855 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074778.112020:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625164160:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.112144:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 256 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.112153:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 256 pid: 8855 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074778.112162:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625164288:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074778.112187:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.118202:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.118416:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625164160:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074778.118430:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625164288:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.118470:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074778.119710:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.121915:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616900352 00000100:00100000:1.0:1769074778.121932:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616900352 00000100:00100000:1.0:1769074778.122081:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616900608 00000100:00100000:1.0:1769074778.122085:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616900608 00000100:00100000:2.0:1769074778.122143:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4686 00000100:00100000:2.0:1769074778.122157:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:60221:x1855007616900352:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:2.0:1769074778.123227:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:60221:x1855007616900352:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 1056us (5018us total) trans 8589943983 rc 0/0 00000100:00100000:2.0:1769074778.123238:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4686 00000100:00080000:3.0:1769074778.125946:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074778.126111:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.126254:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616900736 00000100:00100000:3.0:1769074778.126264:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616900736 00010000:00010000:3.0:1769074778.126301:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 12 pid: 8855 timeout: 1865 lvb_type: 0 00000100:00100000:3.0:1769074778.126346:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7102 00000100:00100000:3.0:1769074778.126354:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616900736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.126370:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.126386:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 12 pid: 8855 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.126397:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 12 pid: 8855 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.126419:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd5337e lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b722 expref: 12 pid: 8855 timeout: 1865 lvb_type: 0 00010000:00010000:3.0:1769074778.126436:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074778.127017:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.127029:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.127045:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.127056:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.127117:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616900736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 762us (1011us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.127125:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7102 00000100:00100000:1.0:1769074778.136059:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4687 00000100:00100000:1.0:1769074778.136075:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:57730:x1855007616900608:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:1.0:1769074778.136385:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+258:57730:x1855007616900608:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 302us (16670us total) trans 8589943984 rc 0/0 00000100:00100000:1.0:1769074778.136393:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4687 00000100:00100000:2.0:1769074778.141843:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.141993:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616900864 00000100:00100000:0.0:1769074778.142008:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616900864 00010000:00010000:0.0:1769074778.142143:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 258 pid: 8855 timeout: 1865 lvb_type: 0 00000100:00100000:0.0:1769074778.142251:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7103 00000100:00100000:0.0:1769074778.142264:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+258:2407:x1855007616900864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.142288:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.142296:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 258 pid: 8855 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074778.142311:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 258 pid: 8855 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074778.142342:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3e00/0x7fe2323adfd53369 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b71b expref: 258 pid: 8855 timeout: 1865 lvb_type: 0 00010000:00010000:0.0:1769074778.142361:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074778.142366:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x48 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.142387:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.142397:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.142449:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.142515:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:2407:x1855007616900864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 252us (685us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.142523:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7103 00010000:00010000:1.0:1769074778.142804:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143335:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074778.143393:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074778.143397:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074778.143420:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143433:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143460:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143467:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143472:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143479:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143484:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0fa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074778.143487:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143497:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143508:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074778.143510:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074778.143514:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143520:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5410e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143535:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143540:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143546:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143550:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074778.143553:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143558:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143576:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b665615e00) 00010000:00010000:1.0:1769074778.143578:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074778.143581:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143586:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54115 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.143621:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54107 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.143635:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.143641:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074778.143644:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074778.143646:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f600, rc 0) 00000100:00100000:1.0:1769074778.143833:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62993:x1855007616900096:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 33297us (33534us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074778.143842:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13338 00000100:00100000:2.0:1769074778.153662:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.153701:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.153728:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.153749:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.153794:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.153850:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616900992 00000100:00100000:2.0:1769074778.153861:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616900992 00000100:00100000:2.0:1769074778.153929:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616901120 00000100:00100000:2.0:1769074778.153932:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616901120 00000100:00100000:2.0:1769074778.153960:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616901248 00000100:00100000:2.0:1769074778.153964:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616901248 00000100:00100000:2.0:1769074778.153979:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616901376 00000100:00100000:2.0:1769074778.153981:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616901376 00000100:00100000:2.0:1769074778.153995:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616901504 00000100:00100000:2.0:1769074778.153997:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616901504 00000100:00100000:2.0:1769074778.154015:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14202 00000100:00100000:2.0:1769074778.154025:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616900992:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074778.154054:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.154074:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c083 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.154102:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154104:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154114:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154131:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154145:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fbe00, rc 0) 00000100:00100000:2.0:1769074778.154201:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616900992:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 178us (550us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.154209:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14202 00000100:00100000:2.0:1769074778.154239:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14203 00000100:00100000:2.0:1769074778.154247:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616901120:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074778.154254:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.154264:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c08a expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.154278:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154280:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154285:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154295:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154303:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fa400, rc 0) 00000100:00100000:2.0:1769074778.154325:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616901120:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 80us (624us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.154331:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14203 00000100:00100000:2.0:1769074778.154344:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14204 00000100:00100000:2.0:1769074778.154351:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616901248:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074778.154357:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.154364:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c091 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.154374:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154376:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154381:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154391:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154399:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fb800, rc 0) 00000100:00100000:2.0:1769074778.154419:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616901248:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 68us (690us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.154428:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14204 00000100:00100000:2.0:1769074778.154440:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14205 00000100:00100000:2.0:1769074778.154447:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616901376:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074778.154454:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.154461:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c098 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.154472:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154474:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.154480:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154492:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.154500:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5491fba00, rc 0) 00000100:00100000:2.0:1769074778.154522:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616901376:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 75us (772us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.154528:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14205 00000100:00100000:2.0:1769074778.154540:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14206 00000100:00100000:2.0:1769074778.154546:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616901504:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074778.154553:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.154563:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd5413f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c09f expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074778.154604:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625164416:192.168.204.60@tcp:106: 00000100:00100000:2.0:1769074778.154652:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00080000:3.0:1769074778.154967:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074778.154971:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074778.160471:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.160835:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616901632 00000100:00100000:1.0:1769074778.160846:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616901632 00000100:00100000:1.0:1769074778.160908:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14207 00000100:00100000:1.0:1769074778.160919:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616901632:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.160937:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.160956:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54146 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0a6 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074778.161018:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625164544:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074778.161122:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b543624b40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074778.174743:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625164416:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074778.174785:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd5413f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c09f expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.174835:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd5413f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c09f expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.174848:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b719800, rc 0) 00000100:00100000:3.0:1769074778.174913:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616901504:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 20365us (21117us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074778.174922:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14206 00000100:00100000:0.0:1769074778.175927:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625164544:192.168.204.60@tcp:106: 00010000:00010000:0.0:1769074778.175958:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54146 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0a6 expref: 11 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074778.175975:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54146 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0a6 expref: 11 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074778.175987:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:0.0:1769074778.176047:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616901632:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 15129us (15588us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074778.176054:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14207 00000100:00100000:2.0:1769074778.183451:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.183865:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616901888 00000100:00100000:1.0:1769074778.183875:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616901888 00000100:00100000:1.0:1769074778.183945:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13339 00000100:00100000:1.0:1769074778.183955:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62993:x1855007616901888:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.183979:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.184004:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5414d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0ad expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.184126:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.184143:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.184152:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.184192:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.184206:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 9 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.184228:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625164672:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.184324:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b543624b40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074778.188077:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.188181:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902016 00000100:00100000:0.0:1769074778.188191:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902016 00010000:00010000:0.0:1769074778.188257:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:0.0:1769074778.188303:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7104 00000100:00100000:0.0:1769074778.188313:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616902016:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.188329:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.188336:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.188351:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.188380:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074778.188385:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.188398:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.188405:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 13 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.188426:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.188495:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616902016:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 183us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.188503:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7104 00010000:00010000:1.0:1769074778.189317:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd5411c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c07c expref: 9 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.189336:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625164672:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.189362:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.189380:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.189387:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074778.189407:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074778.189561:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd5414d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ad expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.189583:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ad expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.189592:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074778.189594:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074778.189597:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:1.0:1769074778.189647:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616901888:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 5694us (6218us total) trans 8589943985 rc 0/0 00000100:00100000:1.0:1769074778.189654:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13339 00000100:00100000:3.0:1769074778.198514:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.198628:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902272 00000100:00100000:1.0:1769074778.198638:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902272 00000100:00100000:1.0:1769074778.198703:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13340 00000100:00100000:1.0:1769074778.198714:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616902272:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.198740:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.198743:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074778.198769:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54154 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ad expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.198789:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074778.198794:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074778.198872:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54162 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0b4 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.198975:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd54169 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.198991:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd54169 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.199002:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd54169 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074778.199015:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00010000:00010000:1.0:1769074778.199114:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f800/0x7fe2323adfd54162 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 12 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0b4 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.199127:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd54169 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0b4 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.199133:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074778.199136:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074778.199138:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f600, rc 0) 00000100:00100000:1.0:1769074778.199193:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616902272:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 481us (689us total) trans 8589943986 rc 0/0 00000100:00100000:1.0:1769074778.199201:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13340 00000100:00100000:0.0:1769074778.204173:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.204327:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902400 00000100:00100000:3.0:1769074778.204339:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902400 00000100:00100000:3.0:1769074778.204409:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13341 00000100:00100000:3.0:1769074778.204420:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616902400:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074778.204459:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074778.204482:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54177 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0bb expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.204548:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54177 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0bb expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.204576:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54177 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0bb expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.204597:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074778.204600:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074778.204604:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867c00, rc 0) 00000100:00100000:3.0:1769074778.204669:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62993:x1855007616902400:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 251us (506us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.204677:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13341 00000100:00100000:0.0:1769074778.212297:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.212416:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902528 00000100:00100000:1.0:1769074778.212429:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902528 00000100:00100000:1.0:1769074778.212489:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14208 00000100:00100000:1.0:1769074778.212499:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616902528:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:1.0:1769074778.212543:0:8396:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.212569:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.212577:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.212585:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.212593:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.212595:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.212599:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 6/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.212635:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 12 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.212648:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 12 pid: 9048 timeout: 1866 lvb_type: 1 00000100:00100000:1.0:1769074778.212664:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625164800:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.212753:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.212762:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 11 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:1.0:1769074778.212778:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625164928:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.212843:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 20 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.212853:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 20 pid: 27843 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.212861:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625165056:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.212890:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.216384:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.216471:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902656 00000100:00100000:3.0:1769074778.216482:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902656 00000100:00100000:3.0:1769074778.216538:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14209 00000100:00100000:3.0:1769074778.216547:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616902656:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:3.0:1769074778.216603:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.216624:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.216633:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.216642:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.216649:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.216652:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.216655:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 6/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.216690:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.216702:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 12 pid: 8396 timeout: 1866 lvb_type: 1 00000100:00100000:3.0:1769074778.216718:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625165184:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.216853:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.216864:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 10 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:3.0:1769074778.216873:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625165312:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.216907:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.216916:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.216929:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625165440:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074778.216952:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03000 going to sleep for 11 seconds 00000100:00100000:0.0:1769074778.223670:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.223713:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.223743:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.223773:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.223857:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.223884:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.223912:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902784 00000100:00100000:0.0:1769074778.223922:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902784 00000100:00100000:0.0:1769074778.223966:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903040 00000100:00100000:0.0:1769074778.223969:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903040 00000100:00100000:0.0:1769074778.223983:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616902912 00000100:00100000:0.0:1769074778.223985:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616902912 00000100:00100000:0.0:1769074778.223996:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903168 00000100:00100000:0.0:1769074778.223998:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903168 00000100:00100000:0.0:1769074778.224016:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14210 00000100:00100000:0.0:1769074778.224025:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616902784:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:0.0:1769074778.224071:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.224093:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 12 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.224101:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074778.224108:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 15 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074778.224115:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074778.224117:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074778.224120:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 6/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.224153:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 15 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074778.224165:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 15 pid: 8399 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.224180:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625165568:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.224284:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074778.224294:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 12 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.224308:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625165696:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.224339:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 12 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.224348:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 12 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:0.0:1769074778.224358:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542faa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625165824:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074778.224394:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074778.224415:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903424 00000100:00100000:0.0:1769074778.224419:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903424 00010000:00010000:0.0:1769074778.224439:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 14 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.224459:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903296 00000100:00100000:0.0:1769074778.224462:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903296 00010000:00010000:0.0:1769074778.224474:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 13 pid: 9048 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.224489:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7105 00000100:00100000:0.0:1769074778.224497:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616903424:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.224507:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.224527:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 14 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.224544:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 14 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.224566:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.224591:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616903424:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 94us (732us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.224598:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7105 00000100:00100000:0.0:1769074778.224614:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7106 00000100:00100000:0.0:1769074778.224620:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616903296:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.224627:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.224634:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 13 pid: 9048 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.224644:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 13 pid: 9048 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.224655:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.224673:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616903296:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 53us (788us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.224679:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7106 00000100:00100000:1.0:1769074778.224990:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14211 00000100:00100000:1.0:1769074778.225004:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616903040:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:1.0:1769074778.225029:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.225042:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.225050:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.225056:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:1.0:1769074778.225058:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:1.0:1769074778.225061:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 5/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.225093:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.225104:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.225113:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625165952:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.225155:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 20 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.225165:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 20 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.225173:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625166080:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.225194:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00010000:00010000:1.0:1769074778.225253:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd53b6c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bcbd expref: 12 pid: 9048 timeout: 1866 lvb_type: 1 00000100:00100000:1.0:1769074778.225272:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625164800:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.225284:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbe00/0x7fe2323adfd54123 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c083 expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:1.0:1769074778.225292:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625164928:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074778.225386:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14212 00000100:00100000:2.0:1769074778.225397:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616902912:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:2.0:1769074778.225420:0:10284:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225432:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 11 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225452:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.225460:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.225466:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.225467:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.225469:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 6/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225490:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.225498:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 20 pid: 27844 timeout: 1866 lvb_type: 1 00000100:00100000:2.0:1769074778.225507:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625166208:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074778.225540:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 12 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.225548:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 12 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:2.0:1769074778.225557:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625166336:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074778.225595:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 11 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225604:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 11 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:2.0:1769074778.225612:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625166464:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074778.225629:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074778.225648:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14213 00000100:00100000:2.0:1769074778.225656:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616903168:12345-192.168.204.60@tcp:1:mirror_io.0 00010000:00010000:2.0:1769074778.225676:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225687:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225698:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074778.225699:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074778.225702:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225721:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 15 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.225729:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 15 pid: 28954 timeout: 1866 lvb_type: 0 00000100:00100000:2.0:1769074778.225738:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625166592:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074778.225756:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:0.0:1769074778.235119:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.235165:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.235262:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903552 00000100:00100000:1.0:1769074778.235272:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903552 00010000:00010000:1.0:1769074778.235314:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:1.0:1769074778.235377:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903680 00000100:00100000:1.0:1769074778.235381:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903680 00010000:00010000:1.0:1769074778.235399:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.235425:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7107 00000100:00100000:1.0:1769074778.235434:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616903552:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074778.235450:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074778.235471:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:1.0:1769074778.235482:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:1.0:1769074778.235509:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074778.235569:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616903552:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 125us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074778.235578:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7107 00000100:00100000:1.0:1769074778.235598:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7108 00000100:00100000:1.0:1769074778.235606:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616903680:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074778.235613:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074778.235622:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.235637:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.235652:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074778.235671:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616903680:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 65us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074778.235677:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7108 00010000:00010000:3.0:1769074778.235965:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fa400/0x7fe2323adfd5412a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c08a expref: 12 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:3.0:1769074778.235978:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625165312:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.235990:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd53fb7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf80 expref: 15 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.235997:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625165440:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074778.249030:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.249081:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.249112:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903808 00000100:00100000:2.0:1769074778.249121:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903808 00010000:00010000:2.0:1769074778.249160:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 15 pid: 8399 timeout: 1866 lvb_type: 1 00000100:00100000:2.0:1769074778.249195:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7109 00000100:00100000:2.0:1769074778.249205:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616903808:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.249231:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074778.249254:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 15 pid: 8399 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.249272:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 15 pid: 8399 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.249299:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074778.249352:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616903808:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 148us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.249360:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7109 00000100:00100000:3.0:1769074778.249536:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904064 00000100:00100000:3.0:1769074778.249541:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904064 00002000:00100000:3.0:1769074778.249587:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3829760->4018175) req@ffff89b546f92680 x1855007616904064/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074778.249597:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92680 with x1855007616904064 ext(3829760->4018175) 00010000:00010000:3.0:1769074778.249604:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.249617:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 12 pid: 27844 timeout: 1866 lvb_type: 0 00002000:00010000:3.0:1769074778.249625:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f92680 00000100:00100000:3.0:1769074778.249643:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 808 00000100:00100000:3.0:1769074778.249652:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616904064:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074778.249848:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.249918:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.250009:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.250043:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616903936 00000100:00100000:3.0:1769074778.250048:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616903936 00010000:00010000:3.0:1769074778.250066:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 12 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:3.0:1769074778.250085:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904320 00000100:00100000:3.0:1769074778.250088:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904320 00010000:00010000:3.0:1769074778.250100:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 13 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.250118:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7110 00000100:00100000:3.0:1769074778.250125:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616903936:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.250135:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.250147:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 12 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:3.0:1769074778.250158:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 12 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:3.0:1769074778.250176:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.250203:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616903936:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 78us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.250210:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7110 00000100:00100000:3.0:1769074778.250236:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7111 00000100:00100000:3.0:1769074778.250243:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616904320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.250251:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.250259:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 13 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.250269:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 13 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.250283:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.250285:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.250290:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.250300:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.250320:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.250342:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616904320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 99us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.250348:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7111 00000100:00100000:1.0:1769074778.251247:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625165952:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.251269:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625166080:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.251290:0:8398:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074778.251504:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625166336:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074778.251513:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625166464:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074778.251534:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904192 00000100:00100000:2.0:1769074778.251540:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904192 00002000:00100000:2.0:1769074778.251655:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3088384->3145727) req@ffff89b67ff0fb80 x1855007616904192/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074778.251663:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0fb80 with x1855007616904192 ext(3088384->3145727) 00010000:00010000:2.0:1769074778.251669:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:2.0:1769074778.251680:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 1866 lvb_type: 0 00002000:00010000:2.0:1769074778.251687:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67ff0fb80 00000100:00100000:2.0:1769074778.251755:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 809 00000100:00100000:2.0:1769074778.251765:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616904192:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:3.0:1769074778.252125:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625165184:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.252152:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074778.252187:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.252206:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.252238:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.252289:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904448 00000100:00100000:2.0:1769074778.252294:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904448 00010000:00010000:2.0:1769074778.252317:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:2.0:1769074778.252343:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904704 00000100:00100000:2.0:1769074778.252347:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904704 00010000:00010000:2.0:1769074778.252359:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 13 pid: 8396 timeout: 1866 lvb_type: 1 00000100:00100000:2.0:1769074778.252769:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7112 00000100:00100000:2.0:1769074778.252779:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616904448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.252790:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074778.252830:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.252841:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.252860:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fba00/0x7fe2323adfd54138 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c098 expref: 13 pid: 27843 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.252875:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074778.252907:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2407:x1855007616904448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 130us (720us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.252913:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7112 00000100:00100000:2.0:1769074778.252931:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7113 00000100:00100000:2.0:1769074778.252937:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2407:x1855007616904704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.252944:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074778.252956:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 13 pid: 8396 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.252964:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 13 pid: 8396 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.252977:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd53fe8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf95 expref: 13 pid: 8396 timeout: 1866 lvb_type: 1 00010000:00010000:2.0:1769074778.252986:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.252988:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.252992:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.253001:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.253039:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074778.253058:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2407:x1855007616904704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 121us (852us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.253064:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7113 00000100:00100000:2.0:1769074778.253115:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625166592:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074778.253129:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253505:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b641651e00/0x7fe2323adfd53fa2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf6b expref: 14 pid: 8399 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.253522:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625165568:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.253533:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5491fb800/0x7fe2323adfd54131 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c091 expref: 11 pid: 27843 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.253541:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625165696:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.253551:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54773a200/0x7fe2323adfd53f6a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bf4f expref: 11 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:0.0:1769074778.253559:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542faa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625165824:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074778.253583:0:36516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253590:0:36516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253596:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253617:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253625:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253638:0:36516:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253645:0:36516:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba44000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074778.253648:0:36516:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253657:0:36516:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253670:0:36516:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.253676:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd5418c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074778.253744:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616902784:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 29720us (30085us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.253752:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14210 00010000:00010000:2.0:1769074778.255117:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255134:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255162:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255171:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255180:0:27843:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255188:0:27843:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074778.255191:0:27843:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255200:0:27843:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255230:0:27843:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.255238:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54185 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074778.255325:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616902656:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 38775us (38959us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.255335:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14209 00000100:00100000:3.0:1769074778.257617:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.257754:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.258140:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625165056:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.258172:0:8396:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074778.258714:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904832 00000100:00100000:3.0:1769074778.258721:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904832 00010000:00010000:3.0:1769074778.258749:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.258772:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904960 00000100:00100000:3.0:1769074778.258774:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904960 00010000:00010000:3.0:1769074778.258782:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 25 pid: 27843 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.258794:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7114 00000100:00100000:3.0:1769074778.258799:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+16:2407:x1855007616904832:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.258833:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.258849:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.258857:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.258876:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877800/0x7fe2323adfd5400b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfb8 expref: 16 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.258890:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.258933:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+15:2407:x1855007616904832:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 135us (1325us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.258938:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7114 00000100:00100000:3.0:1769074778.258952:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7115 00000100:00100000:3.0:1769074778.258955:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e667850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616904960:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.258961:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.258967:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 25 pid: 27843 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.258974:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 25 pid: 27843 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.258982:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9a00/0x7fe2323adfd53b2d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bc9a expref: 25 pid: 27843 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.258989:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.258990:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.258993:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259000:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259013:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.259025:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e667850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007616904960:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 69us (1273us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.259029:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7115 00010000:00010000:3.0:1769074778.259137:0:8396:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259143:0:8396:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259159:0:8396:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259164:0:8396:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259170:0:8396:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259174:0:8396:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074778.259177:0:8396:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259182:0:8396:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259189:0:8396:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.259197:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd5417e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074778.259260:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616902528:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 46758us (46968us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.259269:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14208 00000100:00100000:1.0:1769074778.259293:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616904576 00000100:00100000:1.0:1769074778.259299:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616904576 00000100:00100000:3.0:1769074778.259346:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:10284:1855007625166208:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074778.259366:0:10284:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00002000:00100000:1.0:1769074778.259549:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2998271) req@ffff89b67ff0f800 x1855007616904576/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:314/0 lens 488/0 e 0 to 0 dl 1769074789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074778.259558:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0f800 with x1855007616904576 ext(2097152->2998271) 00010000:00010000:1.0:1769074778.259565:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 14 pid: 28954 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.259576:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 14 pid: 28954 timeout: 1866 lvb_type: 0 00002000:00010000:1.0:1769074778.259583:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67ff0f800 00000100:00100000:1.0:1769074778.259625:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 810 00000100:00100000:1.0:1769074778.259632:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2407:x1855007616904576:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074778.261127:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2409:x1855007616904064:12345-192.168.204.60@tcp:4:cat.0 Request processed in 11464us (12097us total) trans 17179874529 rc 0/0 00000100:00100000:0.0:1769074778.261141:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 808 00002000:00100000:0.0:1769074778.261154:0:8403:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3829760->4018175) req@ffff89b546f92680 x1855007616904064/t17179874529(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:314/0 lens 488/448 e 0 to 0 dl 1769074789 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074778.261164:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b546f92680 with x1855007616904064 ext(3829760->4018175) 00010000:00010000:0.0:1769074778.261173:0:8403:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.261184:0:8403:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 1866 lvb_type: 0 00002000:00010000:0.0:1769074778.261191:0:8403:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b546f92680 00000100:00100000:2.0:1769074778.261340:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616904192:12345-192.168.204.60@tcp:4:cat.0 Request processed in 9575us (11490us total) trans 17179874530 rc 0/0 00000100:00100000:2.0:1769074778.261348:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 809 00002000:00100000:2.0:1769074778.261354:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (3088384->3145727) req@ffff89b67ff0fb80 x1855007616904192/t17179874530(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:314/0 lens 488/448 e 0 to 0 dl 1769074789 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074778.261363:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0fb80 with x1855007616904192 ext(3088384->3145727) 00010000:00010000:2.0:1769074778.261369:0:16437:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 22 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:2.0:1769074778.261378:0:16437:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 22 pid: 8396 timeout: 1866 lvb_type: 0 00002000:00010000:2.0:1769074778.261384:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67ff0fb80 00000100:00100000:2.0:1769074778.271227:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.271718:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616905088 00000100:00100000:0.0:1769074778.271730:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616905088 00010000:00010000:0.0:1769074778.271771:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 22 pid: 27844 timeout: 1866 lvb_type: 1 00000100:00100000:0.0:1769074778.271837:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7116 00000100:00100000:0.0:1769074778.271848:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616905088:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.271862:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.271883:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 22 pid: 27844 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.271900:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 22 pid: 27844 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.271925:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54043 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879bfcd expref: 22 pid: 27844 timeout: 1866 lvb_type: 1 00010000:00010000:0.0:1769074778.271940:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074778.271942:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074778.271946:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.271956:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.271986:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.272033:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616905088:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 187us (831us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.272040:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7116 00010000:00010000:2.0:1769074778.272116:0:10284:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272125:0:10284:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272146:0:10284:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272154:0:10284:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272162:0:10284:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272169:0:10284:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b718000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074778.272173:0:10284:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272181:0:10284:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272194:0:10284:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.272201:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5419a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074778.272267:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616902912:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 46872us (48522us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.272275:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14212 00000100:00100000:2.0:1769074778.281729:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.281780:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.281834:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616905216 00000100:00100000:2.0:1769074778.281844:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616905216 00010000:00010000:2.0:1769074778.281883:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 1866 lvb_type: 0 00000100:00100000:2.0:1769074778.281918:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616905344 00000100:00100000:2.0:1769074778.281921:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616905344 00010000:00010000:2.0:1769074778.281934:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 1866 lvb_type: 0 00000100:00100000:2.0:1769074778.281954:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7117 00000100:00100000:2.0:1769074778.281962:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2408:x1855007616905216:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.281978:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1769074778.281998:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7118 00010000:00010000:2.0:1769074778.281999:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.282008:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616905344:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.282011:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.282016:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074778.282784:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45200/0x7fe2323adfd540dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [3829760->18446744073709551615] (req 3829760->4018175) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c059 expref: 13 pid: 27844 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.284431:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.284450:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.284475:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fae00/0x7fe2323adfd540ac lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [3088384->3145727] (req 3088384->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c021 expref: 21 pid: 8396 timeout: 1866 lvb_type: 0 00010000:00010000:1.0:1769074778.284493:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.284495:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.284500:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.284510:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074778.286300:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616904576:12345-192.168.204.60@tcp:4:cat.0 Request processed in 26511us (33904us total) trans 21474841207 rc 0/0 00010000:00010000:2.0:1769074778.286306:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.286317:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 810 00002000:00100000:0.0:1769074778.286331:0:24027:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->2998271) req@ffff89b67ff0f800 x1855007616904576/t21474841207(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:314/0 lens 488/448 e 0 to 0 dl 1769074789 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1769074778.286335:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2408:x1855007616905216:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4374us (4620us total) trans 0 rc 0/0 00002000:00100000:0.0:1769074778.286342:0:24027:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0f800 with x1855007616904576 ext(2097152->2998271) 00000100:00100000:2.0:1769074778.286345:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7117 00010000:00010000:0.0:1769074778.286350:0:24027:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.286367:0:24027:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 1866 lvb_type: 0 00002000:00010000:0.0:1769074778.286375:0:24027:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff89b67ff0f800 00010000:00010000:3.0:1769074778.286691:0:8398:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286703:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286730:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286738:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286746:0:8398:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286758:0:8398:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0fa00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074778.286762:0:8398:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286771:0:8398:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286788:0:8398:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074778.286794:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54193 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.286798:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.286907:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616905344:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4899us (5125us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.286915:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7118 00000100:00100000:3.0:1769074778.288621:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616903040:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 63607us (64893us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.288636:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14211 00000100:00100000:3.0:1769074778.291521:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.291584:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616905472 00000100:00100000:0.0:1769074778.291665:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616905472 00010000:00010000:0.0:1769074778.291902:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 1866 lvb_type: 0 00000100:00100000:0.0:1769074778.291945:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7119 00000100:00100000:0.0:1769074778.291957:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616905472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.291978:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.292005:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.292017:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.292040:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd540cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->2998271) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c044 expref: 16 pid: 28954 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.292058:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074778.292060:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074778.292065:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.292075:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.292105:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074778.292132:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292141:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292161:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292169:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292176:0:28955:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292183:0:28955:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b718600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074778.292187:0:28955:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292194:0:28955:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292206:0:28955:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074778.292225:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd541a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074778.292286:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616905472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 330us (775us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.292295:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7119 00000100:00100000:2.0:1769074778.292326:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2409:x1855007616903168:12345-192.168.204.60@tcp:1:mirror_io.0 Request processed in 66671us (68552us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.292336:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14213 00000100:00100000:0.0:1769074778.294931:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.295026:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616905728 00000100:00100000:1.0:1769074778.295036:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616905728 00000100:00100000:1.0:1769074778.295096:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13342 00000100:00100000:1.0:1769074778.295107:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62993:x1855007616905728:12345-192.168.204.60@tcp:36:mirror_io.0 00010000:00010000:1.0:1769074778.295268:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074778.295291:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54177 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0bb expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295304:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074778.295309:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074778.295335:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.295345:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295351:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 256 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295355:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295359:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 256 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295363:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295368:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295374:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 9/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.295410:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295423:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.295447:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625166720:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.295535:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295543:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.295551:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625166848:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.295582:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 256 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295589:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 256 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.295610:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625166976:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.295638:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295648:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.295661:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625167104:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.295701:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 256 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295709:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 256 pid: 6513 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.295717:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625167232:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074778.295748:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.295756:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 10 pid: 6512 timeout: 1866 lvb_type: 0 00000100:00100000:1.0:1769074778.295764:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625167360:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.295786:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074778.298893:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625166720:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.298914:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625166848:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074778.303300:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.303328:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.303486:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616905984 00000100:00100000:1.0:1769074778.303486:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625166976:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074778.303495:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616905984 00000100:00100000:1.0:1769074778.303499:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625167104:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.303507:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625167232:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074778.303514:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625167360:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074778.303545:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616906240 00000100:00100000:0.0:1769074778.303549:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616906240 00010000:00010000:1.0:1769074778.303555:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074778.303576:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4688 00000100:00100000:0.0:1769074778.303585:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:46917:x1855007616905984:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:0.0:1769074778.303848:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+258:46917:x1855007616905984:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 262us (557us total) trans 8589943987 rc 0/0 00000100:00100000:0.0:1769074778.303856:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4688 00000100:00100000:0.0:1769074778.303873:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4689 00000100:00100000:0.0:1769074778.303880:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:39340:x1855007616906240:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:0.0:1769074778.303954:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+12:39340:x1855007616906240:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 73us (625us total) trans 8589943988 rc 0/0 00000100:00100000:0.0:1769074778.303960:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4689 00000100:00100000:3.0:1769074778.304141:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.304168:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.304188:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.304469:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.304492:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616906368 00000100:00100000:3.0:1769074778.304497:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616906368 00010000:00010000:3.0:1769074778.304521:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 258 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.304548:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616906624 00000100:00100000:3.0:1769074778.304551:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616906624 00010000:00010000:3.0:1769074778.304640:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 259 pid: 6513 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.304664:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616906496 00000100:00100000:3.0:1769074778.304668:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616906496 00010000:00010000:3.0:1769074778.304682:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 12 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.304691:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616906752 00000100:00100000:3.0:1769074778.304693:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616906752 00010000:00010000:3.0:1769074778.304703:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 13 pid: 6512 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.304715:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7120 00000100:00100000:3.0:1769074778.304721:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+259:2406:x1855007616906368:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.304732:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.304737:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 259 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.304748:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 259 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.304768:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd52e37 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 13 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3f6 expref: 259 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.304784:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074778.304789:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.304799:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.304853:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+258:2406:x1855007616906368:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 131us (711us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.304860:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7120 00000100:00100000:3.0:1769074778.304875:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7121 00000100:00100000:3.0:1769074778.304882:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+257:2406:x1855007616906624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.304889:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.304895:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 257 pid: 6513 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.304903:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 257 pid: 6513 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.304965:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd533af lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b737 expref: 257 pid: 6513 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.304975:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074778.304979:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.304985:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.305005:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+256:2406:x1855007616906624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 124us (837us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.305012:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7121 00000100:00100000:3.0:1769074778.305025:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7122 00000100:00100000:3.0:1769074778.305032:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e667050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616906496:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.305040:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.305044:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 13 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:2.0:1769074778.305326:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7123 00000100:00100000:2.0:1769074778.305337:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+13:2406:x1855007616906752:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074778.305346:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074778.305350:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 13 pid: 6512 timeout: 1866 lvb_type: 0 00010000:00010000:2.0:1769074778.305359:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 13 pid: 6512 timeout: 1866 lvb_type: 0 00010000:00010000:2.0:1769074778.305385:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd531be lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1a/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b61f expref: 13 pid: 6512 timeout: 1866 lvb_type: 0 00010000:00010000:2.0:1769074778.305396:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074778.305399:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074778.305405:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074778.305442:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2406:x1855007616906752:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 108us (974us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.305449:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7123 00010000:00010000:3.0:1769074778.305630:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 11 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.305643:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd52e30 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b3ef expref: 11 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.305654:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074778.305658:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.305665:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.305686:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e667050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616906496:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 654us (1497us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.305693:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7122 00000100:00100000:0.0:1769074778.312266:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.312380:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907008 00000100:00100000:3.0:1769074778.312393:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907008 00010000:00010000:3.0:1769074778.312440:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:3.0:1769074778.312524:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7124 00000100:00100000:3.0:1769074778.312534:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+256:2408:x1855007616907008:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074778.312551:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074778.312558:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.312583:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.312772:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54910d400/0x7fe2323adfd52fe9 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b507 expref: 256 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:3.0:1769074778.312793:0:6499:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074778.312799:0:6499:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074778.312839:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074778.312916:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+255:2408:x1855007616907008:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 382us (666us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.312924:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7124 00000100:00100000:1.0:1769074778.316755:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.316913:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616906880 00000100:00100000:3.0:1769074778.316923:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616906880 00010000:00010000:2.0:1769074778.317004:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00000100:00100000:0.0:1769074778.317067:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7125 00000100:00100000:0.0:1769074778.317080:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616906880:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074778.317095:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074778.317103:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.317117:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.317140:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd52f6b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879b4ba expref: 10 pid: 6514 timeout: 1866 lvb_type: 0 00010000:00010000:0.0:1769074778.317155:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074778.317159:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.317167:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.317172:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.317239:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074778.317285:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616906880:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 209us (539us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.317290:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7125 00010000:00010000:1.0:1769074778.317916:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074778.317926:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000040:00080000:1.0:1769074778.318191:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2411, 88 off206384 00000004:00080000:1.0:1769074778.318198:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766160390777ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318281:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2411, 88 off206120 00000004:00080000:1.0:1769074778.318289:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766160482520ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318317:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2412, 88 off206208 00000004:00080000:1.0:1769074778.318321:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766160514202ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318331:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2412, 88 off206472 00000004:00080000:1.0:1769074778.318334:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766160527649ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318343:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2413, 88 off206560 00000004:00080000:1.0:1769074778.318347:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766160539840ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318355:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2413, 88 off206296 00000004:00080000:1.0:1769074778.318358:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766160551110ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318365:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2414, 88 off206384 00000004:00080000:1.0:1769074778.318369:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766160561912ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318391:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2415, 88 off206472 00000004:00080000:1.0:1769074778.318395:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766160588619ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318406:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2414, 88 off206648 00000004:00080000:1.0:1769074778.318410:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766160602780ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318419:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2416, 88 off206560 00000004:00080000:1.0:1769074778.318423:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766160615950ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318433:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2415, 88 off206736 00000004:00080000:1.0:1769074778.318794:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766160629280ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318835:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2417, 88 off206648 00000004:00080000:1.0:1769074778.318841:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766161033507ns, next at 1706953247299ns, rc = 0 00000040:00080000:1.0:1769074778.318865:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2416, 88 off206824 00000004:00080000:1.0:1769074778.318869:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766161062615ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318878:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2417, 88 off206912 00000004:00080000:1.0:1769074778.318881:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766161074115ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318889:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2418, 88 off207000 00000004:00080000:1.0:1769074778.318892:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766161085109ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318899:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2419, 88 off207088 00000004:00080000:1.0:1769074778.318903:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1766161095910ns, next at 1706951295843ns, rc = 0 00000040:00080000:1.0:1769074778.318912:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2418, 88 off206736 00000004:00080000:1.0:1769074778.318916:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1766161108693ns, next at 1706953247299ns, rc = 0 00000001:00080000:0.0:1769074778.321792:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943989 is committed 00000004:00080000:0.0:1769074778.321830:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1922 committed 00000004:00080000:0.0:1769074778.321847:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1922 committed 00000004:00080000:0.0:1769074778.321865:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1921 committed 00000004:00080000:0.0:1769074778.321867:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1920 committed 00000004:00080000:0.0:1769074778.321870:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1919 committed 00000004:00080000:0.0:1769074778.321871:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1921 committed 00000004:00080000:0.0:1769074778.321874:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1918 committed 00000004:00080000:0.0:1769074778.321876:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1920 committed 00000004:00080000:0.0:1769074778.321878:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1917 committed 00000004:00080000:0.0:1769074778.321879:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1919 committed 00000004:00080000:0.0:1769074778.321881:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1918 committed 00000004:00080000:0.0:1769074778.321883:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1917 committed 00000004:00080000:0.0:1769074778.321885:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1916 committed 00000004:00080000:0.0:1769074778.321886:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1915 committed 00000004:00080000:0.0:1769074778.321889:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1916 committed 00000004:00080000:0.0:1769074778.321891:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1915 committed 00000004:00080000:0.0:1769074778.321892:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1914 committed 00000001:00080000:0.0:1769074778.321898:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943988 is committed 00000001:00080000:0.0:1769074778.321906:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943987 is committed 00000004:00010000:0.0:1769074778.322029:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000a9a7f5ae reply state 000000005871c2b0 transno 8589943989 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.322054:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.322064:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074778.322067:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:0.0:1769074778.322072:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074778.322234:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625167872:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.322338:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625167744:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074778.322394:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625167488:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.322755:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074778.322790:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625168256:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.322881:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074778.322923:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625167744 00000100:00100000:2.0:1769074778.322931:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625167744 00000100:00100000:2.0:1769074778.322973:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168256 00000100:00100000:2.0:1769074778.322975:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168256 00000100:00100000:2.0:1769074778.322990:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14214 00000100:00100000:2.0:1769074778.322997:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625167744:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.323164:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625167744:12345-0@lo:2:osp-syn-1-0.0 Request processed in 167us (410us total) trans 17179874531 rc 0/0 00000100:00100000:2.0:1769074778.323172:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14214 00000100:00100000:2.0:1769074778.323187:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14215 00000100:00100000:2.0:1769074778.323193:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625168256:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074778.323254:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074778.323257:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.323259:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.323262:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.323264:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:2.0:1769074778.323285:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625168256:12345-0@lo:2:osp-syn-1-0.0 Request processed in 92us (406us total) trans 17179874532 rc 0/0 00000100:00100000:2.0:1769074778.323290:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14215 00000100:00080000:2.0:1769074778.323414:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074778.323519:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625168768:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074778.323544:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074778.323574:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625169280:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074778.323602:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1769074778.323637:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d4a80/2, rc 0, transno 17179874531 00000100:00100000:2.0:1769074778.323646:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625167744:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074778.323654:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d5f80/2, rc 0, transno 17179874532 00000100:00100000:2.0:1769074778.323656:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625168256:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.323662:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1769074778.323666:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625168896:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074778.323680:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074778.323690:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625169408:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074778.323700:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074778.323717:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168768 00000100:00100000:2.0:1769074778.323720:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168768 00000100:00100000:2.0:1769074778.323740:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169280 00000100:00100000:2.0:1769074778.323742:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169280 00000100:00100000:2.0:1769074778.323750:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168896 00000100:00100000:2.0:1769074778.323751:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168896 00000100:00100000:2.0:1769074778.323759:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169408 00000100:00100000:2.0:1769074778.323760:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169408 00000100:00100000:2.0:1769074778.323771:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14216 00000100:00100000:2.0:1769074778.323774:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625168768:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074778.323899:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625168768:12345-0@lo:2:osp-syn-0-0.0 Request processed in 123us (353us total) trans 21474841208 rc 0/0 00000100:00100000:2.0:1769074778.323905:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14216 00000100:00100000:2.0:1769074778.323915:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14217 00000100:00100000:2.0:1769074778.323920:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625169280:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074778.323973:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:2.0:1769074778.323976:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:2.0:1769074778.323978:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.323980:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:2.0:1769074778.323982:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:2.0:1769074778.324001:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625169280:12345-0@lo:2:osp-syn-0-0.0 Request processed in 81us (398us total) trans 21474841209 rc 0/0 00000100:00100000:2.0:1769074778.324007:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14217 00000100:00100000:2.0:1769074778.324016:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14218 00000100:00100000:2.0:1769074778.324021:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625168896:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074778.324066:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074778.324068:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.324070:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.324072:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.324074:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:2.0:1769074778.324098:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625168896:12345-0@lo:2:osp-syn-0-0.0 Request processed in 76us (415us total) trans 21474841210 rc 0/0 00000100:00100000:2.0:1769074778.324104:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14218 00000100:00100000:2.0:1769074778.324116:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14219 00000100:00100000:2.0:1769074778.324126:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625169408:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074778.324172:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074778.324174:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:2.0:1769074778.324177:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.324178:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:2.0:1769074778.324180:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074778.324204:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625169408:12345-0@lo:2:osp-syn-0-0.0 Request processed in 79us (497us total) trans 21474841211 rc 0/0 00000100:00100000:2.0:1769074778.324208:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14219 00000100:00080000:2.0:1769074778.324241:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:2.0:1769074778.324277:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f93100/2, rc 0, transno 21474841210 00000100:00100000:2.0:1769074778.324281:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625168896:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074778.324292:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f91180/2, rc 0, transno 21474841211 00000100:00100000:2.0:1769074778.324296:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625169408:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074778.324301:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f93800/2, rc 0, transno 21474841208 00000100:00100000:2.0:1769074778.324304:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625168768:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074778.324309:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f91c00/2, rc 0, transno 21474841209 00000100:00100000:2.0:1769074778.324312:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625169280:0@lo:2:osp-syn-0-0.0 00010000:00080000:0.0:1769074778.324822:0:6514:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:0.0:1769074778.324841:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616905728:12345-192.168.204.60@tcp:36:mirror_io.0 Request processed in 29732us (29917us total) trans 8589943989 rc 0/0 00000100:00100000:0.0:1769074778.324849:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13342 00010000:00010000:0.0:1769074778.324878:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.324887:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.324894:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.324900:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ec00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074778.324904:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.324912:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.324928:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074778.324931:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074778.324936:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074778.324942:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541a8 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074778.325304:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.325341:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625168000:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074778.325368:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.325394:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625168512:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074778.325411:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.325420:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625167616:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074778.325435:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625168384:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074778.325438:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.325455:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625168128:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074778.325467:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.325477:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625167488 00000100:00100000:3.0:1769074778.325483:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625167488 00000100:00100000:3.0:1769074778.326639:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168000 00000100:00100000:3.0:1769074778.326648:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168000 00000100:00100000:1.0:1769074778.327146:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625167872 00000100:00100000:1.0:1769074778.327152:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625167872 00000100:00100000:1.0:1769074778.327182:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625167616 00000100:00100000:1.0:1769074778.327186:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625167616 00000100:00100000:1.0:1769074778.327203:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168384 00000100:00100000:1.0:1769074778.327205:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168384 00000100:00100000:1.0:1769074778.327237:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14220 00000100:00100000:1.0:1769074778.327246:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3651:x1855007625167488:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.327483:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14221 00000100:00100000:2.0:1769074778.327494:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625168000:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.327518:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14222 00000100:00100000:2.0:1769074778.327524:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625167872:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074778.327584:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:2.0:1769074778.327588:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.327600:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.327603:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.327606:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074778.328034:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14223 00000100:00100000:0.0:1769074778.328045:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625167616:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074778.328183:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625167488:12345-0@lo:2:osp-syn-1-0.0 Request processed in 937us (2878us total) trans 17179874533 rc 0/0 00000100:00100000:2.0:1769074778.328183:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625167872:12345-0@lo:2:osp-syn-1-0.0 Request processed in 657us (2770us total) trans 17179874534 rc 0/0 00000100:00100000:2.0:1769074778.328191:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14222 00000100:00100000:1.0:1769074778.328193:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14220 00000100:00100000:1.0:1769074778.328209:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14224 00000100:00100000:1.0:1769074778.328225:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625168384:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074778.328287:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb size from disk: 8388608 -> 8388608 00002000:00010000:1.0:1769074778.328290:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074778.328292:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074778.328294:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:1.0:1769074778.328296:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb blocks from disk: 8192 -> 8192 00000100:00100000:1.0:1769074778.328323:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625168384:12345-0@lo:2:osp-syn-1-0.0 Request processed in 109us (2855us total) trans 17179874536 rc 0/0 00000100:00100000:3.0:1769074778.328362:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.328397:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625168640:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074778.328430:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.328441:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625169152:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074778.328466:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074778.328494:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168128 00000100:00100000:3.0:1769074778.328500:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168128 00000100:00100000:1.0:1769074778.329305:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14224 00000100:00100000:1.0:1769074778.329333:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168640 00000100:00100000:1.0:1769074778.329338:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168640 00000100:00100000:1.0:1769074778.329359:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169152 00000100:00100000:1.0:1769074778.329362:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169152 00000100:00100000:1.0:1769074778.329386:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14225 00000100:00100000:1.0:1769074778.329393:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625168640:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074778.329577:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625168640:12345-0@lo:2:osp-syn-0-0.0 Request processed in 184us (1145us total) trans 21474841212 rc 0/0 00000100:00100000:1.0:1769074778.329583:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14225 00000100:00100000:1.0:1769074778.329595:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14226 00000100:00100000:1.0:1769074778.329601:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625169152:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074778.329654:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074778.329657:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074778.329661:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074778.329663:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:1.0:1769074778.329665:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074778.329690:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625169152:12345-0@lo:2:osp-syn-0-0.0 Request processed in 90us (1223us total) trans 21474841213 rc 0/0 00000100:00100000:1.0:1769074778.329695:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14226 00000100:00100000:1.0:1769074778.329778:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074778.330284:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625167616:12345-0@lo:2:osp-syn-1-0.0 Request processed in 2242us (4843us total) trans 17179874535 rc 0/0 00000100:00100000:0.0:1769074778.330293:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14223 00000100:00100000:0.0:1769074778.330319:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907136 00000100:00100000:0.0:1769074778.330325:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907136 00000100:00100000:0.0:1769074778.330375:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13343 00000100:00100000:0.0:1769074778.330383:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62993:x1855007616907136:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074778.330399:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074778.330419:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0c2 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.330477:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.330492:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074778.330509:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074778.330511:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074778.330514:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44000, rc 0) 00000100:00100000:0.0:1769074778.330640:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616907136:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 176us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074778.330647:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13343 00000004:00080000:0.0:1769074778.330956:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d4380/2, rc 0, transno 17179874535 00000100:00100000:0.0:1769074778.330963:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625167616:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074778.330978:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f92300/2, rc 0, transno 21474841212 00000100:00100000:0.0:1769074778.330983:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625168640:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074778.330992:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f90e00/2, rc 0, transno 21474841213 00000100:00100000:0.0:1769074778.330996:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625169152:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074778.331021:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074778.331161:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625169024:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074778.331198:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074778.331309:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625168512 00000100:00100000:2.0:1769074778.331317:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625168512 00000100:00100000:2.0:1769074778.331357:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14227 00000100:00100000:2.0:1769074778.331378:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625168128:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074778.331524:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074778.331528:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:2.0:1769074778.331530:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.331533:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:2.0:1769074778.331534:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074778.331588:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625168128:12345-0@lo:2:osp-syn-1-0.0 Request processed in 224us (3225us total) trans 17179874537 rc 0/0 00000100:00100000:2.0:1769074778.331595:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14227 00000100:00100000:2.0:1769074778.331610:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14228 00000100:00100000:2.0:1769074778.331616:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3651:x1855007625168512:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074778.331699:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625168512:12345-0@lo:2:osp-syn-0-0.0 Request processed in 83us (677us total) trans 21474841214 rc 0/0 00000100:00100000:2.0:1769074778.331705:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14228 00000004:00080000:2.0:1769074778.331743:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d5180/2, rc 0, transno 17179874537 00000100:00100000:2.0:1769074778.331748:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625168128:0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074778.331851:0:28954:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074778.331854:0:28954:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.331856:0:28954:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074778.331858:0:28954:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074778.331861:0:28954:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074778.331888:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625168000:12345-0@lo:2:osp-syn-1-0.0 Request processed in 4397us (6518us total) trans 17179874538 rc 0/0 00000100:00100000:2.0:1769074778.331893:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14221 00000004:00080000:2.0:1769074778.331924:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d5c00/2, rc 0, transno 17179874534 00000100:00100000:2.0:1769074778.331929:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625167872:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074778.331939:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545389880/2, rc 0, transno 17179874536 00000100:00100000:2.0:1769074778.331942:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625168384:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074778.332434:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625169536:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074778.332464:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1769074778.332491:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d7b80/2, rc 0, transno 17179874533 00000100:00100000:0.0:1769074778.332496:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625167488:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074778.332504:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5436d4700/2, rc 0, transno 17179874538 00000100:00100000:0.0:1769074778.332508:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625168000:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1769074778.332520:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f90000/2, rc 0, transno 21474841214 00000100:00100000:0.0:1769074778.332523:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625168512:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074778.332541:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169024 00000100:00100000:0.0:1769074778.332545:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169024 00000100:00100000:0.0:1769074778.332583:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169536 00000100:00100000:0.0:1769074778.332586:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169536 00000100:00100000:0.0:1769074778.332604:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14229 00000100:00100000:0.0:1769074778.332611:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+12:3651:x1855007625169024:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074778.332682:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:0.0:1769074778.332684:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074778.332687:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074778.332688:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074778.332690:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074778.332721:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625169024:12345-0@lo:2:osp-syn-0-0.0 Request processed in 110us (1522us total) trans 21474841215 rc 0/0 00000100:00100000:0.0:1769074778.332727:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14229 00000100:00100000:0.0:1769074778.332741:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14230 00000100:00100000:0.0:1769074778.332747:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625169536:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074778.332826:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074778.332829:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074778.332832:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074778.332834:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074755 00002000:00010000:0.0:1769074778.332836:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074778.332861:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625169536:12345-0@lo:2:osp-syn-1-0.0 Request processed in 114us (396us total) trans 17179874539 rc 0/0 00000100:00100000:0.0:1769074778.332866:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14230 00000004:00080000:0.0:1769074778.332895:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f92a00/2, rc 0, transno 21474841215 00000100:00100000:0.0:1769074778.332900:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625169024:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074778.332907:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f93b80/2, rc 0, transno 17179874539 00000100:00100000:0.0:1769074778.332910:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625169536:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074778.337553:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.337839:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907264 00000100:00100000:1.0:1769074778.337851:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907264 00000100:00100000:1.0:1769074778.337907:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13344 00000100:00100000:1.0:1769074778.337916:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616907264:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.337938:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.337957:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd541b6 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0c9 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.338054:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd541bd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074778.338070:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd541bd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074778.338081:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd541bd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074778.338115:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 16777217 00000004:00200000:1.0:1769074778.338119:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074778.338137:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd541bd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074778.338271:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd541b6 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c9 expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.338300:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd541bd lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c9 expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074778.338309:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074778.338312:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074778.338316:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fc00, rc 0) 00000100:00100000:1.0:1769074778.338509:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616907264:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 593us (1487us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074778.338517:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13344 00000100:00100000:0.0:1769074778.343543:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.344252:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907392 00000100:00100000:3.0:1769074778.344264:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907392 00000100:00100000:3.0:1769074778.344331:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14231 00000100:00100000:3.0:1769074778.344342:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616907392:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074778.344363:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074778.344389:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866600/0x7fe2323adfd541c4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0d0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074778.344416:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bc:0x0] initial LVB size: 1048576, mtime: 0x6971f054, atime: 0x6971f052, ctime: 0x6971f054, blocks: 0x7e8 00010000:00010000:3.0:1769074778.344437:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.344440:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.344449:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866600/0x7fe2323adfd541c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0d0 expref: 10 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.344467:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866600/0x7fe2323adfd541c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0d0 expref: 10 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.344482:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866600, rc 0) 00000100:00100000:3.0:1769074778.344554:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616907392:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 214us (1021us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.344562:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14231 00000100:00100000:2.0:1769074778.345957:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074778.346180:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907520 00000100:00100000:1.0:1769074778.346189:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907520 00000100:00100000:1.0:1769074778.346245:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14232 00000100:00100000:1.0:1769074778.346254:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2408:x1855007616907520:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.346269:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1769074778.346286:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074778.346288:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541cb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0d7 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074778.346311:0:10284:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2ba:0x0] initial LVB size: 2097152, mtime: 0x6971f058, atime: 0x6971f058, ctime: 0x6971f058, blocks: 0x800 00010000:00010000:1.0:1769074778.346326:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.346328:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.346337:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541cb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0d7 expref: 9 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.346351:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541cb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0d7 expref: 9 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.346368:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ec00, rc 0) 00000100:00100000:3.0:1769074778.347008:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907648 00000100:00100000:3.0:1769074778.347015:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907648 00000100:00100000:1.0:1769074778.347153:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616907520:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 887us (1189us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074778.347162:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14232 00000100:00100000:1.0:1769074778.347185:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14233 00000100:00100000:1.0:1769074778.347193:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616907648:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:1.0:1769074778.347202:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074778.347226:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd541d2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0de expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074778.347241:0:10284:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bb:0x0] initial LVB size: 2097152, mtime: 0x6971f058, atime: 0x6971f058, ctime: 0x6971f057, blocks: 0x800 00010000:00010000:1.0:1769074778.347253:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.347255:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074778.347262:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd541d2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0de expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.347275:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd541d2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0de expref: 10 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074778.347285:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0ee00, rc 0) 00000100:00100000:1.0:1769074778.347317:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616907648:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 125us (1031us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074778.347324:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14233 00000100:00100000:2.0:1769074778.348504:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.348909:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907776 00000100:00100000:3.0:1769074778.348922:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907776 00000100:00100000:3.0:1769074778.348983:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14234 00000100:00100000:3.0:1769074778.348993:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616907776:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074778.349009:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074778.349029:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866c00/0x7fe2323adfd541d9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0e5 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074778.349050:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2be:0x0] initial LVB size: 2097152, mtime: 0x6971f058, atime: 0x6971f058, ctime: 0x6971f058, blocks: 0x808 00010000:00010000:3.0:1769074778.349069:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.349071:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.349080:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866c00/0x7fe2323adfd541d9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0e5 expref: 11 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.349097:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866c00/0x7fe2323adfd541d9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0e5 expref: 11 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.349110:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866c00, rc 0) 00000100:00100000:3.0:1769074778.349528:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616907776:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 534us (1027us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.349536:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14234 00000100:00100000:0.0:1769074778.350168:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074778.350455:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616907904 00000100:00100000:3.0:1769074778.350464:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616907904 00000100:00100000:3.0:1769074778.350511:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14235 00000100:00100000:3.0:1769074778.350520:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616907904:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074778.350536:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074778.350557:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0ec expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074778.350873:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bf:0x0] initial LVB size: 4805728, mtime: 0x6971f059, atime: 0x6971f059, ctime: 0x6971f059, blocks: 0x14b0 00010000:00010000:3.0:1769074778.350887:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.350888:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074778.350896:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 12 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.350907:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 12 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074778.350916:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867800, rc 0) 00000100:00100000:3.0:1769074778.350958:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616907904:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 440us (795us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074778.350962:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14235 00000100:00100000:2.0:1769074778.351461:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074778.351960:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908032 00000100:00100000:2.0:1769074778.351968:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908032 00000100:00100000:2.0:1769074778.352020:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14236 00000100:00100000:2.0:1769074778.352029:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616908032:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074778.352043:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074778.352062:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd541e7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0f3 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074778.352083:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bc:0x0] initial LVB size: 4194304, mtime: 0x6971f059, atime: 0x6971f058, ctime: 0x6971f059, blocks: 0x1008 00010000:00010000:2.0:1769074778.352099:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.352101:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074778.352110:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd541e7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0f3 expref: 11 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.352125:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd541e7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0f3 expref: 11 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074778.352135:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446800, rc 0) 00000100:00100000:2.0:1769074778.352321:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616908032:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 293us (862us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074778.352328:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14236 00010000:00010000:2.0:1769074778.525083:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074779.037387:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa23100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625169664:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074779.037466:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.037893:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169664 00000100:00100000:1.0:1769074779.037902:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169664 00000100:00100000:1.0:1769074779.037959:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2649 00000100:00100000:1.0:1769074779.037971:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625169664:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074779.038035:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6438912 pending 0 free 1446150144 avail 1299910656 00000100:00100000:1.0:1769074779.038103:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625169664:12345-0@lo:41:osp-pre-1-0.0 Request processed in 135us (638us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.038111:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2649 00000100:00100000:1.0:1769074779.038696:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa23100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625169664:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074779.038909:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625169792:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074779.038953:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074779.039044:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169792 00000100:00100000:2.0:1769074779.039051:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169792 00000100:00100000:2.0:1769074779.039099:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2650 00000100:00100000:2.0:1769074779.039106:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625169792:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074779.039351:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 6389760 pending 0 free 1445400576 avail 1299161088 00000100:00100000:2.0:1769074779.039420:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625169792:12345-0@lo:41:osp-pre-0-1.0 Request processed in 314us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.039427:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2650 00000100:00100000:0.0:1769074779.039577:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625169792:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074779.039920:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625170176:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.039957:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.039995:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625170688:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040032:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.040055:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1769074779.040061:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625170048:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040080:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.040094:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa23100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625170560:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040114:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.040324:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170688 00000100:00100000:1.0:1769074779.040330:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170688 00000100:00100000:1.0:1769074779.040379:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170048 00000100:00100000:1.0:1769074779.040383:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170048 00000100:00100000:1.0:1769074779.040401:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170560 00000100:00100000:1.0:1769074779.040403:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170560 00000100:00100000:1.0:1769074779.040418:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2323 00000100:00100000:1.0:1769074779.040424:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625170688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040471:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625170688:12345-0@lo:400:kworker.0 Request processed in 46us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.040478:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2323 00000100:00100000:1.0:1769074779.040491:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2324 00000100:00100000:1.0:1769074779.040497:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625170048:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040521:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625170048:12345-0@lo:400:kworker.0 Request processed in 25us (440us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.040525:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2324 00000100:00100000:1.0:1769074779.040534:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2325 00000100:00100000:1.0:1769074779.040539:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625170560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040559:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625170560:12345-0@lo:400:kworker.0 Request processed in 21us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.040565:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2325 00000100:00100000:1.0:1769074779.040762:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625170048:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040772:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa23100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625170560:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040780:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625170688:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040837:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625169920:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040870:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.040898:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625170432:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040919:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.040935:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625170944:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040954:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.040970:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1769074779.040976:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625170304:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.040997:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.041014:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625170816:0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.041152:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170176 00000100:00100000:3.0:1769074779.041159:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170176 00000100:00100000:3.0:1769074779.041198:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170304 00000100:00100000:3.0:1769074779.041230:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170304 00000100:00100000:3.0:1769074779.041255:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1452 00000100:00100000:3.0:1769074779.041263:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625170176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.041329:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625170176:12345-0@lo:400:kworker.0 Request processed in 67us (1371us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074779.041339:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1452 00000100:00100000:3.0:1769074779.041353:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1453 00000100:00100000:3.0:1769074779.041370:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625170304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.041401:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625170304:12345-0@lo:400:kworker.0 Request processed in 32us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074779.041407:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1453 00000100:00100000:3.0:1769074779.041437:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625170176:0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.041477:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170432 00000100:00100000:3.0:1769074779.041481:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170432 00000100:00100000:3.0:1769074779.041502:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170944 00000100:00100000:3.0:1769074779.041505:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170944 00000100:00100000:3.0:1769074779.041514:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2326 00000100:00100000:3.0:1769074779.041519:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625170432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.041549:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625170432:12345-0@lo:400:kworker.0 Request processed in 30us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074779.041555:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2326 00000100:00100000:3.0:1769074779.041566:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2327 00000100:00100000:3.0:1769074779.041572:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625170944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.042519:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625170944:12345-0@lo:400:kworker.0 Request processed in 941us (1558us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074779.042528:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2327 00000100:00100000:0.0:1769074779.042725:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625169920 00000100:00100000:0.0:1769074779.042732:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625169920 00000100:00100000:0.0:1769074779.042767:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1769074779.042775:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625169920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074779.042832:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625169920:12345-0@lo:400:kworker.0 Request processed in 58us (1961us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.042837:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00080000:2.0:1769074779.042981:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074779.042989:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074779.042992:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074779.042995:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074779.043297:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.043344:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625169920:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.043552:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625170432:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.043569:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625170944:0@lo:400:kworker.0 00000100:00100000:1.0:1769074779.043695:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625170304:0@lo:400:kworker.0 00000100:00100000:2.0:1769074779.043777:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625170816 00000100:00100000:2.0:1769074779.043782:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625170816 00000100:00100000:2.0:1769074779.043876:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2328 00000100:00100000:2.0:1769074779.043884:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625170816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074779.043928:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625170816:12345-0@lo:400:kworker.0 Request processed in 46us (631us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.043934:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2328 00000100:00080000:0.0:1769074779.044005:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074779.044269:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625170816:0@lo:400:kworker.0 00000100:00100000:3.0:1769074779.423617:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.423677:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.423708:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.423733:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908160 00000100:00100000:2.0:1769074779.423746:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908160 00000100:00100000:0.0:1769074779.423880:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908416 00000100:00100000:0.0:1769074779.423885:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908416 00000100:00100000:2.0:1769074779.424037:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14237 00000100:00100000:2.0:1769074779.424048:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616908160:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:2.0:1769074779.424155:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616908160:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 110us (553us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.424163:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14237 00000100:00100000:2.0:1769074779.424181:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14238 00000100:00100000:2.0:1769074779.424187:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616908416:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:2.0:1769074779.424245:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616908416:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 58us (568us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.424252:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14238 00000100:00100000:3.0:1769074779.428926:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.429094:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.429240:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.429484:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908672 00000100:00100000:1.0:1769074779.429502:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908672 00000100:00100000:1.0:1769074779.429557:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908288 00000100:00100000:1.0:1769074779.429561:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908288 00000100:00100000:1.0:1769074779.429585:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908544 00000100:00100000:1.0:1769074779.429588:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908544 00000100:00100000:1.0:1769074779.429609:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14239 00000100:00100000:1.0:1769074779.429619:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616908672:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074779.429751:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616908672:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 133us (6040us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.429758:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14239 00000100:00100000:1.0:1769074779.429778:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14240 00000100:00100000:1.0:1769074779.429784:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616908288:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074779.429863:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616908288:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 79us (949us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.429869:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14240 00000100:00100000:1.0:1769074779.429882:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14241 00000100:00100000:1.0:1769074779.429889:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616908544:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074779.429918:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616908544:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 30us (825us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.429925:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14241 00000100:00100000:2.0:1769074779.431064:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908800 00000100:00100000:2.0:1769074779.431077:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908800 00000100:00100000:2.0:1769074779.431125:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14242 00000100:00100000:2.0:1769074779.431135:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616908800:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:2.0:1769074779.431243:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616908800:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 109us (2002us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.431251:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14242 00000100:00100000:1.0:1769074779.435830:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.435927:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616908928 00000100:00100000:0.0:1769074779.435937:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616908928 00000100:00100000:0.0:1769074779.436031:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14243 00000100:00100000:0.0:1769074779.436040:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616908928:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074779.436153:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616908928:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 113us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.436160:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14243 00000100:00100000:3.0:1769074779.437896:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.438595:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909056 00000100:00100000:0.0:1769074779.438606:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909056 00000100:00100000:0.0:1769074779.438887:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14244 00000100:00100000:0.0:1769074779.438898:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616909056:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074779.440444:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616909056:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 1538us (2547us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.440460:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14244 00000100:00100000:3.0:1769074779.442610:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.443462:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.443930:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.444101:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909184 00000100:00100000:0.0:1769074779.444111:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909184 00000100:00100000:0.0:1769074779.444171:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909312 00000100:00100000:0.0:1769074779.444174:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909312 00000100:00100000:0.0:1769074779.444210:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14245 00000100:00100000:0.0:1769074779.444221:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616909184:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074779.444323:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616909184:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 104us (1724us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.444332:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14245 00000100:00100000:0.0:1769074779.444350:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14246 00000100:00100000:0.0:1769074779.444356:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616909312:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:0.0:1769074779.444408:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616909312:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 52us (1549us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.444414:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14246 00000100:00100000:3.0:1769074779.445287:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.445413:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909568 00000100:00100000:1.0:1769074779.445423:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909568 00000100:00100000:1.0:1769074779.445472:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909440 00000100:00100000:1.0:1769074779.445476:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909440 00000100:00100000:1.0:1769074779.445500:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14247 00000100:00100000:1.0:1769074779.445508:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616909568:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074779.445594:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616909568:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 87us (1767us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.445601:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14247 00000100:00100000:1.0:1769074779.445616:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14248 00000100:00100000:1.0:1769074779.445622:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616909440:12345-192.168.204.60@tcp:23:mirror_io.0 00000100:00100000:1.0:1769074779.445662:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616909440:12345-192.168.204.60@tcp:23:mirror_io.0 Request processed in 39us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.445668:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14248 00000100:00100000:0.0:1769074779.455664:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.456178:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909696 00000100:00100000:2.0:1769074779.456189:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909696 00000100:00100000:2.0:1769074779.456264:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14249 00000100:00100000:2.0:1769074779.456272:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62993:x1855007616909696:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074779.456289:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074779.456306:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c0fa expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.456337:0:8399:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.456346:0:8399:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.456364:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.456366:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.456370:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 5/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.456402:0:8399:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.456416:0:8399:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 10 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.456430:0:8399:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8399:1855007625171072:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074779.456539:0:8399:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.456553:0:8399:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 14 pid: 27844 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.456564:0:8399:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8399:1855007625171200:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074779.456592:0:8399:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074779.460861:0:8399:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8399:1855007625171072:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074779.460876:0:8399:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8399:1855007625171200:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074779.460909:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.460930:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446200, rc 0) 00000100:00100000:0.0:1769074779.461035:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616909696:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 4762us (5381us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.461049:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14249 00000100:00100000:3.0:1769074779.461937:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.462032:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.462246:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909952 00000100:00100000:1.0:1769074779.462254:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909952 00010000:00010000:1.0:1769074779.462287:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 15 pid: 27844 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.462317:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616909824 00000100:00100000:1.0:1769074779.462320:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616909824 00010000:00010000:1.0:1769074779.462346:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 11 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.462364:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7126 00000100:00100000:1.0:1769074779.462372:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616909952:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074779.462386:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074779.462406:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 15 pid: 27844 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.462416:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 15 pid: 27844 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.462442:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c800/0x7fe2323adfd52301 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad04 expref: 15 pid: 27844 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.462457:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074779.462497:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2409:x1855007616909952:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 125us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.462504:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7126 00000100:00100000:1.0:1769074779.462520:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7127 00000100:00100000:1.0:1769074779.462526:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616909824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074779.462537:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074779.462546:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 11 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.462556:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 11 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.462568:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910c200/0x7fe2323adfd5234e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad35 expref: 11 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.462577:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.462579:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.462586:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 11 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.462595:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 11 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.462627:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 11 pid: 8399 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.462641:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625171328:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074779.462689:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442c480 going to sleep for 11 seconds 00000100:00080000:1.0:1769074779.462707:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074779.470594:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625171328:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074779.470628:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074779.470701:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616909824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8173us (8667us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.470713:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7127 00000100:00100000:2.0:1769074779.483109:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.484259:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910080 00000100:00100000:0.0:1769074779.484272:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910080 00000100:00100000:0.0:1769074779.484372:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8656 00000100:00100000:0.0:1769074779.484382:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616910080:12345-192.168.204.60@tcp:22:mirror_io.0 00002000:00010000:0.0:1769074779.484622:0:24027:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074779.484626:0:24027:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074779.484629:0:24027:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074779.485070:0:24027:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:0.0:1769074779.485072:0:24027:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:0.0:1769074779.485149:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616910080:12345-192.168.204.60@tcp:22:mirror_io.0 Request processed in 767us (2050us total) trans 21474841216 rc 0/0 00000100:00100000:0.0:1769074779.485157:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8656 00000100:00100000:0.0:1769074779.513024:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.513130:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910208 00000100:00100000:2.0:1769074779.513141:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910208 00000100:00100000:2.0:1769074779.513216:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14250 00000100:00100000:2.0:1769074779.513225:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616910208:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074779.513245:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074779.513270:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c101 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.513312:0:8399:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.513322:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.513324:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.513329:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 13 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.513366:0:8399:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 19 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.513382:0:8399:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 19 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.513397:0:8399:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8399:1855007625171456:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074779.513496:0:8399:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074779.528059:0:8399:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8399:1855007625171456:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074779.528096:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 13 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.528116:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67fea0400, rc 0) 00000100:00100000:2.0:1769074779.528369:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62993:x1855007616910208:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 15142us (15354us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.528378:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14250 00000100:00100000:0.0:1769074779.528875:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.528984:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910336 00000100:00100000:1.0:1769074779.528991:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910336 00010000:00010000:1.0:1769074779.529074:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 20 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.529117:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7128 00000100:00100000:1.0:1769074779.529126:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616910336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074779.529143:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074779.529166:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 20 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.529206:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 20 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.529235:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910d600/0x7fe2323adfd52347 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad2e expref: 20 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.529248:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.529250:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.529257:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.529268:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.529301:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.529315:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625171584:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074779.529392:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442c480 going to sleep for 11 seconds 00000100:00080000:0.0:1769074779.530027:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074779.532237:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.532428:0:24027:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910464 00000100:00100000:2.0:1769074779.532437:0:24027:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910464 00000100:00100000:2.0:1769074779.532508:0:24027:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8657 00000100:00100000:2.0:1769074779.532518:0:24027:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616910464:12345-192.168.204.60@tcp:22:mirror_io.0 00002000:00010000:2.0:1769074779.532723:0:24027:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074779.532727:0:24027:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074779.532729:0:24027:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074779.532732:0:24027:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074754 00002000:00010000:2.0:1769074779.532734:0:24027:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:2.0:1769074779.532795:0:24027:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616910464:12345-192.168.204.60@tcp:22:mirror_io.0 Request processed in 278us (567us total) trans 17179874540 rc 0/0 00000100:00100000:2.0:1769074779.532803:0:24027:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8657 00000100:00100000:1.0:1769074779.533791:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625171584:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074779.537958:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074779.538057:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616910336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8922us (9174us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.538069:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7128 00000100:00100000:0.0:1769074779.546453:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.546553:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910592 00000100:00100000:3.0:1769074779.546562:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910592 00000100:00100000:3.0:1769074779.546630:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8658 00000100:00100000:3.0:1769074779.546640:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2409:x1855007616910592:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:3.0:1769074779.546673:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.546687:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.546689:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.546695:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.546708:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.549005:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074779.549917:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625171712:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074779.549983:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.550040:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625171712 00000100:00100000:1.0:1769074779.550046:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625171712 00000100:00100000:1.0:1769074779.550087:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1168 00000100:00100000:1.0:1769074779.550095:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625171712:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074779.550115:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1088836928 pending 2199552 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074779.550159:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625171712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 64us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.550165:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1168 00000100:00100000:1.0:1769074779.550257:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625171712:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074779.550307:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074779.550313:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625171840:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074779.550336:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074779.550369:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625171840 00000100:00100000:1.0:1769074779.550373:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625171840 00000100:00100000:1.0:1769074779.550395:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1169 00000100:00100000:1.0:1769074779.550401:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625171840:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074779.550414:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1088836928 pending 2199552 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074779.550442:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625171840:12345-0@lo:13:osp-pre-0-0.0 Request processed in 41us (105us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.550450:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1169 00000100:00100000:1.0:1769074779.550479:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625171840:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074779.551232:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625171968:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074779.551270:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074779.551380:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625171968 00000100:00100000:2.0:1769074779.551386:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625171968 00000100:00100000:2.0:1769074779.551427:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1170 00000100:00100000:2.0:1769074779.551434:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625171968:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074779.551447:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1094489408 pending 1486848 free 3900067840 avail 3668545536 00000100:00100000:2.0:1769074779.551496:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625171968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.551502:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1170 00000100:00100000:1.0:1769074779.551875:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625171968:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074779.554392:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.555250:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.555283:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.555314:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910848 00000100:00100000:0.0:1769074779.555325:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910848 00000100:00100000:0.0:1769074779.555386:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911104 00000100:00100000:0.0:1769074779.555389:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911104 00000100:00100000:0.0:1769074779.555408:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911360 00000100:00100000:0.0:1769074779.555412:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911360 00000100:00100000:0.0:1769074779.555431:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8659 00000100:00100000:0.0:1769074779.555440:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2409:x1855007616910848:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:0.0:1769074779.555474:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.555490:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.555492:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.555497:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.555511:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074779.556078:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8660 00000100:00100000:0.0:1769074779.556086:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2409:x1855007616911104:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:0.0:1769074779.556108:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.556118:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.556119:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.556123:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.556133:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074779.556630:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.557572:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911616 00000100:00100000:1.0:1769074779.557582:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911616 00000100:00100000:1.0:1769074779.557641:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8661 00000100:00100000:1.0:1769074779.557650:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2409:x1855007616911360:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074779.557678:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.557691:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.557693:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.557697:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.557710:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.558290:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8662 00000100:00100000:1.0:1769074779.558298:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+20:2409:x1855007616911616:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074779.558317:0:28444:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.558326:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.558328:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.558332:0:28444:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.558342:0:28444:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.558727:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.558758:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.558788:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.558844:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.558863:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910720 00000100:00100000:1.0:1769074779.558870:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910720 00000100:00100000:1.0:1769074779.558908:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616910976 00000100:00100000:1.0:1769074779.558911:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616910976 00000100:00100000:1.0:1769074779.558930:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911232 00000100:00100000:1.0:1769074779.558933:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911232 00000100:00100000:1.0:1769074779.558949:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911488 00000100:00100000:1.0:1769074779.558952:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911488 00000100:00100000:1.0:1769074779.558969:0:28446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8663 00000100:00100000:1.0:1769074779.558976:0:28446:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2408:x1855007616910720:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074779.558997:0:28446:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.559009:0:28446:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.559011:0:28446:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.559014:0:28446:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.559025:0:28446:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.559576:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8664 00000100:00100000:1.0:1769074779.559584:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+22:2408:x1855007616910976:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074779.559605:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.559614:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.559615:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.559619:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.559629:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.560241:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8665 00000100:00100000:1.0:1769074779.560250:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2408:x1855007616911232:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074779.560269:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.560278:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.560280:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.560283:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.560292:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.560877:0:24031:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8666 00000100:00100000:1.0:1769074779.560887:0:24031:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2408:x1855007616911488:12345-192.168.204.60@tcp:3:mirror_io.0 00010000:00010000:1.0:1769074779.560908:0:24031:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.560917:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.560918:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.560923:0:24031:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.560932:0:24031:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596767:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596787:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596797:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596845:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074779.596850:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596860:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596879:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.596886:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866000/0x7fe2323adfd541fc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [12288->1048575] (req 12288->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074779.596981:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+22:2409:x1855007616910592:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 50338us (50534us total) trans 0 rc 1036288/1036288 00000100:00100000:3.0:1769074779.596992:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8658 00010000:00010000:0.0:1769074779.608062:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.608082:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.608091:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.608098:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910de00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074779.608102:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.608111:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.608130:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.608137:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54910de00/0x7fe2323adfd54203 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074779.608257:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+19:2409:x1855007616910848:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 52813us (53870us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074779.608268:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8659 00010000:00010000:1.0:1769074779.619109:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.619128:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.619137:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.619143:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54910c600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.619148:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.619156:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.619174:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.619181:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c600/0x7fe2323adfd5420a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.619264:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+20:2409:x1855007616911104:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 63173us (64013us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074779.619274:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8660 00010000:00010000:3.0:1769074779.640188:0:28444:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.641488:0:28444:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.641498:0:28444:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.641510:0:28444:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074779.641514:0:28444:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.641524:0:28444:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.642431:0:28444:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.642445:0:28444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e800/0x7fe2323adfd54218 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074779.643479:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+18:2409:x1855007616911616:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 85171us (86840us total) trans 0 rc 611424/611424 00000100:00100000:2.0:1769074779.643493:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8662 00010000:00010000:1.0:1769074779.646512:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.646542:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.646551:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.646557:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0ea00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.646562:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.646570:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.646901:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.646917:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0ea00/0x7fe2323adfd54211 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.647031:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616911360:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 89376us (91741us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074779.647042:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8661 00010000:00010000:1.0:1769074779.655425:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.655444:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.655453:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.655460:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0f800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.655464:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.655472:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.655490:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.655498:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f800/0x7fe2323adfd54226 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.655600:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616910976:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 96011us (96836us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074779.655611:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8664 00010000:00010000:3.0:1769074779.657492:0:28446:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.657512:0:28446:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.657520:0:28446:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.657527:0:28446:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0fa00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074779.657531:0:28446:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.657546:0:28446:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.657570:0:28446:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.657588:0:28446:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd5421f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074779.657692:0:28446:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+17:2408:x1855007616910720:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 98710us (98960us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074779.657703:0:28446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8663 00010000:00010000:1.0:1769074779.679761:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.679794:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.679827:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.679834:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0fe00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.679839:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.679847:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.679864:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.679871:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd5422d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.679952:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616911232:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 119699us (121158us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074779.679963:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8665 00010000:00010000:1.0:1769074779.687255:0:24031:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.687277:0:24031:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.687287:0:24031:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.687295:0:24031:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.687301:0:24031:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.687310:0:24031:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.687334:0:24031:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.687342:0:24031:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54234 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.687445:0:24031:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616911488:12345-192.168.204.60@tcp:3:mirror_io.0 Request processed in 126555us (128595us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074779.687456:0:24031:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8666 00000100:00100000:2.0:1769074779.706896:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.707002:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911744 00000100:00100000:1.0:1769074779.707013:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911744 00000100:00100000:1.0:1769074779.707091:0:24031:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8667 00000100:00100000:1.0:1769074779.707101:0:24031:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2407:x1855007616911744:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074779.707140:0:24031:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.707160:0:24031:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 13 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.707168:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074779.707170:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074779.707174:0:24031:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.707229:0:24031:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 13 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.707244:0:24031:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 13 pid: 8399 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.707259:0:24031:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_009:lustre-OST0000_UUID:24031:1855007625172096:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074779.707327:0:24031:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984300 going to sleep for 11 seconds 00000100:00100000:1.0:1769074779.711950:0:24031:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_009:lustre-OST0000_UUID:24031:1855007625172096:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074779.711980:0:24031:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074779.712550:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.712797:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616911872 00000100:00100000:1.0:1769074779.712845:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616911872 00010000:00010000:1.0:1769074779.712883:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 14 pid: 8399 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.712922:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7129 00000100:00100000:1.0:1769074779.712931:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616911872:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074779.712946:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074779.712966:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 14 pid: 8399 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.712980:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 14 pid: 8399 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.713001:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446200/0x7fe2323adfd541ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0fa expref: 14 pid: 8399 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.713016:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.713018:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.713023:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.713033:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.713064:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074779.713113:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616911872:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 182us (565us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.713121:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7129 00010000:00010000:3.0:1769074779.713452:0:24031:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.713462:0:24031:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000001:00080000:2.0:1769074779.743560:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841217 is committed 00000001:00080000:2.0:1769074779.743583:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841215 is committed 00000001:00080000:2.0:1769074779.743603:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841207 is committed 00000001:00080000:2.0:1769074779.743609:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841206 is committed 00002000:00010000:3.0:1769074779.744182:0:24031:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074754 -> 1769074779 00002000:00010000:3.0:1769074779.744198:0:24031:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074754 -> 1769074779 00010000:00010000:3.0:1769074779.744215:0:24031:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.744228:0:24031:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.744236:0:24031:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.744243:0:24031:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074779.744249:0:24031:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.744258:0:24031:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.744282:0:24031:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.744289:0:24031:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd5423b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [12288->2097151] (req 12288->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074779.745374:0:24031:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616911744:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 38264us (38478us total) trans 21474841217 rc 0/0 00000100:00100000:3.0:1769074779.745388:0:24031:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8667 00000100:00100000:0.0:1769074779.755270:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.755466:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912000 00000100:00100000:3.0:1769074779.755476:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912000 00000100:00100000:3.0:1769074779.755612:0:24031:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8668 00000100:00100000:3.0:1769074779.755622:0:24031:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616912000:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:3.0:1769074779.755657:0:24031:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.755685:0:24031:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 5 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.755692:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.755694:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.755697:0:24031:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.755732:0:24031:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 5 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.755746:0:24031:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 5 pid: 27844 timeout: 1867 lvb_type: 0 00000100:00100000:3.0:1769074779.755760:0:24031:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109500 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_009:lustre-OST0000_UUID:24031:1855007625172224:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074779.756396:0:24031:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b030c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074779.759224:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.759274:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.759452:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912128 00000100:00100000:3.0:1769074779.759465:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912128 00000100:00100000:3.0:1769074779.759590:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912256 00000100:00100000:3.0:1769074779.759594:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912256 00000100:00100000:3.0:1769074779.759634:0:28446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8669 00000100:00100000:3.0:1769074779.759644:0:28446:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616912128:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:3.0:1769074779.759699:0:28446:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.759723:0:28446:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.759731:0:28446:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.759733:0:28446:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.759737:0:28446:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.759779:0:28446:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.759803:0:28446:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 11 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:3.0:1769074779.760090:0:28446:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_014:lustre-OST0001_UUID:28446:1855007625172352:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074779.760181:0:28446:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03f00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074779.760273:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8670 00000100:00100000:0.0:1769074779.760285:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616912256:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:0.0:1769074779.760305:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.760315:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.760316:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.760319:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.760326:0:8403:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074779.764297:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.764463:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912384 00000100:00100000:2.0:1769074779.764475:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912384 00000100:00100000:2.0:1769074779.764615:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8671 00000100:00100000:2.0:1769074779.764626:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616912384:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:2.0:1769074779.764676:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.764696:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.764700:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.764704:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.764715:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074779.766604:0:24031:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109500 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_009:lustre-OST0000_UUID:24031:1855007625172224:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074779.766643:0:24031:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.766921:0:28446:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_014:lustre-OST0001_UUID:28446:1855007625172352:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074779.766939:0:28446:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074779.771385:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.771563:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.774224:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912512 00000100:00100000:2.0:1769074779.774249:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912512 00010000:00010000:2.0:1769074779.774316:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 6 pid: 27844 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.774370:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912640 00000100:00100000:1.0:1769074779.774375:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912640 00010000:00010000:1.0:1769074779.774412:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.774452:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7130 00000100:00100000:1.0:1769074779.774462:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+6:2409:x1855007616912512:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074779.774478:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074779.774501:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 6 pid: 27844 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.774515:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 6 pid: 27844 timeout: 1867 lvb_type: 0 00000100:00100000:3.0:1769074779.774654:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7131 00000100:00100000:3.0:1769074779.774676:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616912640:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074779.774687:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074779.774702:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074779.774713:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074779.774730:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b682164200/0x7fe2323adfd523b7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad6d expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074779.774751:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.774753:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.774758:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.774768:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.774779:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.774781:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.774783:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.774801:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.774878:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074779.774945:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616912640:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 282us (3392us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074779.774953:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7131 00010000:00010000:3.0:1769074779.774986:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.774995:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.776365:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b64eff3000/0x7fe2323adfd52371 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ad43 expref: 6 pid: 27844 timeout: 1867 lvb_type: 0 00010000:00010000:1.0:1769074779.776388:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.776390:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.776395:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.776404:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.776412:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.776413:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074779.776415:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.776421:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.776466:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074779.776501:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2409:x1855007616912512:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 2039us (5129us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074779.776511:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7130 00010000:00010000:1.0:1769074779.778737:0:28446:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.778756:0:28446:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.779277:0:24031:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.779287:0:24031:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.779486:0:8403:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.779495:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00080000:0.0:1769074779.780389:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:1.0:1769074779.832235:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874542 is committed 00000001:00080000:1.0:1769074779.832261:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874539 is committed 00000001:00080000:1.0:1769074779.832279:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874530 is committed 00000001:00080000:1.0:1769074779.832287:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874529 is committed 00002000:00010000:3.0:1769074779.832491:0:28446:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074754 -> 1769074779 00002000:00010000:3.0:1769074779.832497:0:28446:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074754 -> 1769074779 00010000:00010000:3.0:1769074779.832511:0:28446:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.832524:0:28446:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.832533:0:28446:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.832547:0:28446:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074779.832552:0:28446:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.832571:0:28446:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.832592:0:28446:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.832599:0:28446:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54249 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28446 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074779.832722:0:28446:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616912128:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 73072us (73500us total) trans 17179874542 rc 0/0 00000100:00100000:3.0:1769074779.832734:0:28446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8669 00010000:00010000:0.0:1769074779.833531:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.833542:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.833551:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.833558:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b546117000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074779.833572:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.833581:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.833601:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.833608:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54257 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074779.834995:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616912384:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 70359us (70696us total) trans 17179874541 rc 0/0 00000100:00100000:0.0:1769074779.835008:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8671 00000001:00080000:0.0:1769074779.838617:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841218 is committed 00002000:00010000:1.0:1769074779.838768:0:24031:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074779 00002000:00010000:1.0:1769074779.838773:0:24031:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074779 00010000:00010000:1.0:1769074779.838800:0:24031:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.838840:0:24031:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.838849:0:24031:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.838857:0:24031:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba44600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.838861:0:24031:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.838870:0:24031:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.838890:0:24031:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.838897:0:24031:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd54242 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.838995:0:24031:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616912000:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 83370us (83729us total) trans 21474841218 rc 0/0 00000100:00100000:1.0:1769074779.840486:0:24031:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8668 00000001:00080000:0.0:1769074779.858382:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841219 is committed 00010000:00010000:2.0:1769074779.858739:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.858754:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.858768:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.858776:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074779.858780:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.858789:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.858901:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.858910:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54250 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074779.859024:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616912256:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 98734us (99741us total) trans 21474841219 rc 0/0 00000100:00100000:2.0:1769074779.859036:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8670 00000100:00100000:3.0:1769074779.870718:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.870760:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.872957:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912896 00000100:00100000:2.0:1769074779.872973:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912896 00000100:00100000:2.0:1769074779.873066:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616912768 00000100:00100000:2.0:1769074779.873070:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616912768 00000100:00100000:2.0:1769074779.873116:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8672 00000100:00100000:2.0:1769074779.873126:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2406:x1855007616912896:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:2.0:1769074779.873166:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.873185:0:8403:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.873205:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074779.873207:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074779.873211:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.873249:0:8403:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.873262:0:8403:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 13 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.873275:0:8403:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_001:lustre-OST0001_UUID:8403:1855007625172480:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074779.873365:0:8403:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5449b0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074779.873457:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8673 00000100:00100000:0.0:1769074779.873468:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2407:x1855007616912768:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:0.0:1769074779.873503:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.873515:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.873520:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074779.873522:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074779.873524:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.873552:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.873565:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 11 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:0.0:1769074779.873575:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625172608:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074779.873603:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6656046c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074779.880229:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.880287:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074779.880578:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913152 00000100:00100000:2.0:1769074779.880587:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913152 00010000:00010000:2.0:1769074779.880623:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 12 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.880665:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913024 00000100:00100000:2.0:1769074779.880669:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913024 00010000:00010000:2.0:1769074779.880683:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 14 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.880706:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7132 00000100:00100000:2.0:1769074779.880715:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616913152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074779.880730:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074779.880753:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 12 pid: 36516 timeout: 1867 lvb_type: 0 00010000:00010000:2.0:1769074779.880765:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 12 pid: 36516 timeout: 1867 lvb_type: 0 00010000:00010000:2.0:1769074779.880789:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.880791:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.880796:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.880859:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.880883:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074779.880931:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616913152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 216us (711us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.880940:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7132 00000100:00100000:2.0:1769074779.880959:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7133 00000100:00100000:2.0:1769074779.880966:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616913024:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074779.880974:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074779.880987:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 14 pid: 36516 timeout: 1867 lvb_type: 0 00010000:00010000:2.0:1769074779.880998:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 14 pid: 36516 timeout: 1867 lvb_type: 0 00010000:00010000:2.0:1769074779.881011:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.881013:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074779.881017:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.881023:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.881035:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074779.881055:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616913024:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 90us (769us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074779.881061:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7133 00010000:00010000:2.0:1769074779.881118:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd52275 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acc5 expref: 13 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:2.0:1769074779.881131:0:8403:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_001:lustre-OST0001_UUID:8403:1855007625172480:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074779.881152:0:8403:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.881162:0:8403:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.881169:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.882368:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9e00/0x7fe2323adfd5226e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879acbe expref: 11 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:0.0:1769074779.882386:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625172608:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074779.882404:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.882413:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.882419:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074779.918903:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874543 is committed 00002000:00010000:2.0:1769074779.919871:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074754 -> 1769074779 00002000:00010000:2.0:1769074779.919877:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074754 -> 1769074779 00010000:00010000:2.0:1769074779.919897:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.919910:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.919919:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.919927:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b546117000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074779.919932:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.919940:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.919961:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074779.919969:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd5425e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074779.920078:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2406:x1855007616912896:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 46948us (49365us total) trans 17179874543 rc 0/0 00000100:00100000:2.0:1769074779.920090:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8672 00000001:00080000:0.0:1769074779.922449:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841220 is committed 00002000:00010000:1.0:1769074779.927541:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074754 -> 1769074779 00002000:00010000:1.0:1769074779.927554:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074754 -> 1769074779 00010000:00010000:1.0:1769074779.927570:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.927584:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.927593:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.927600:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.927605:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.927614:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.927634:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.927641:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876400/0x7fe2323adfd54265 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.927768:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2407:x1855007616912768:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 54297us (57001us total) trans 21474841220 rc 0/0 00000100:00100000:1.0:1769074779.927783:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8673 00000100:00100000:0.0:1769074779.945926:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.946030:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913280 00000100:00100000:1.0:1769074779.946041:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913280 00000100:00100000:1.0:1769074779.946121:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8674 00000100:00100000:1.0:1769074779.946131:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2409:x1855007616913280:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074779.946166:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.946195:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.946204:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074779.946206:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074779.946209:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.946243:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.946257:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 10 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.946270:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625172736:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074779.946354:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074779.951737:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074779.951863:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913408 00000100:00100000:3.0:1769074779.951873:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913408 00010000:00010000:3.0:1769074779.951910:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 11 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:3.0:1769074779.951958:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7134 00000100:00100000:3.0:1769074779.951968:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616913408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074779.951984:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074779.952005:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 11 pid: 36516 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074779.952019:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 11 pid: 36516 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074779.952046:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.952048:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074779.952053:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.952063:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074779.952083:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074779.952140:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2409:x1855007616913408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 173us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074779.952148:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7134 00010000:00010000:1.0:1769074779.952557:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5227c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879accc expref: 10 pid: 36516 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.952579:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625172736:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074779.952600:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.952610:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.952618:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000001:00080000:2.0:1769074779.971261:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841221 is committed 00002000:00010000:1.0:1769074779.971388:0:28445:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4808704 00002000:00010000:1.0:1769074779.971393:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074755 -> 1769074779 00002000:00010000:1.0:1769074779.971396:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074755 -> 1769074779 00010000:00010000:1.0:1769074779.971407:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.971418:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.971426:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.971433:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074779.971438:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.971452:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.971471:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.971478:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e400/0x7fe2323adfd5426c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074779.971592:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2409:x1855007616913280:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 25458us (25671us total) trans 21474841221 rc 0/0 00000100:00100000:1.0:1769074779.971602:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8674 00000100:00100000:2.0:1769074779.986665:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074779.986922:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913536 00000100:00100000:1.0:1769074779.986934:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913536 00000100:00100000:1.0:1769074779.987016:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8675 00000100:00100000:1.0:1769074779.987027:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616913536:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074779.987062:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.987081:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.987088:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074779.987090:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074779.987094:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.987132:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 12 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.987145:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 12 pid: 8399 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074779.987159:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0001_UUID:28445:1855007625172864:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074779.987276:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074779.991338:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0001_UUID:28445:1855007625172864:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074779.991371:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074779.993354:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074779.993576:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913664 00000100:00100000:0.0:1769074779.993587:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913664 00010000:00010000:0.0:1769074779.993626:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 13 pid: 8399 timeout: 1867 lvb_type: 0 00000100:00100000:0.0:1769074779.993670:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7135 00000100:00100000:0.0:1769074779.993679:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616913664:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074779.993695:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074779.993716:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 13 pid: 8399 timeout: 1867 lvb_type: 0 00010000:00010000:0.0:1769074779.993729:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 13 pid: 8399 timeout: 1867 lvb_type: 0 00010000:00010000:0.0:1769074779.993752:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67fea0400/0x7fe2323adfd541f5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c101 expref: 13 pid: 8399 timeout: 1867 lvb_type: 0 00010000:00010000:0.0:1769074779.993771:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.993773:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074779.993779:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.993791:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074779.993879:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074779.993940:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616913664:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 262us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074779.993947:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7135 00010000:00010000:1.0:1769074779.994129:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074779.994139:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074780.060474:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874544 is committed 00000100:00100000:3.0:1769074780.061368:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074780.061394:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625172992:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074780.061465:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.061666:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625172992 00000100:00100000:1.0:1769074780.061675:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625172992 00000100:00100000:1.0:1769074780.061724:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1171 00000100:00100000:1.0:1769074780.061733:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625172992:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1769074780.061754:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 1119327552 pending 0 free 3900067840 avail 3659632640 00000100:00100000:1.0:1769074780.061838:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625172992:12345-0@lo:13:osp-pre-1-1.0 Request processed in 106us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.061846:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1171 00000100:00100000:3.0:1769074780.062377:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625172992:0@lo:13:osp-pre-1-1.0 00002000:00010000:0.0:1769074780.063075:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074754 -> 1769074779 00002000:00010000:0.0:1769074780.063080:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074754 -> 1769074779 00010000:00010000:0.0:1769074780.063659:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.063673:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.063682:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.063689:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074780.063694:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.063703:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.063727:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.063735:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e400/0x7fe2323adfd54273 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [12288->4194303] (req 12288->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074780.064329:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616913536:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 76973us (77345us total) trans 17179874544 rc 0/0 00000100:00100000:0.0:1769074780.064342:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8675 00000100:00100000:0.0:1769074780.105091:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.105340:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913792 00000100:00100000:1.0:1769074780.105350:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913792 00000100:00100000:1.0:1769074780.105432:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8676 00000100:00100000:1.0:1769074780.105452:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2406:x1855007616913792:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:1.0:1769074780.105488:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.105505:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.105514:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074780.105516:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074780.105520:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00080000:0.0:1769074780.105725:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1769074780.105957:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.105971:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 11 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:1.0:1769074780.105986:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0001_UUID:28445:1855007625173120:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074780.106284:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:2.0:1769074780.109422:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0001_UUID:28445:1855007625173120:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074780.109454:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074780.112294:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.113559:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616913920 00000100:00100000:3.0:1769074780.113577:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616913920 00010000:00010000:3.0:1769074780.113653:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00000100:00100000:3.0:1769074780.113758:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7136 00000100:00100000:3.0:1769074780.113770:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616913920:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074780.113802:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074780.113873:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074780.113887:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074780.113914:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910dc00/0x7fe2323adfd5229f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace1 expref: 12 pid: 28955 timeout: 1867 lvb_type: 0 00010000:00010000:3.0:1769074780.113933:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.113936:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.113941:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.113951:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.113996:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074780.114191:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.114201:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074780.114937:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2409:x1855007616913920:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1166us (2652us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.114947:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7136 00000001:00080000:0.0:1769074780.169209:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874545 is committed 00002000:00010000:2.0:1769074780.169372:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074755 -> 1769074779 00002000:00010000:2.0:1769074780.169377:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074755 -> 1769074779 00010000:00010000:2.0:1769074780.169394:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.169406:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.169415:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.169423:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f0e200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074780.169427:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.169435:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.169457:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.169464:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0e200/0x7fe2323adfd5427a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074780.169589:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2406:x1855007616913792:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 64142us (64502us total) trans 17179874545 rc 0/0 00000100:00100000:2.0:1769074780.169601:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8676 00000100:00100000:1.0:1769074780.183521:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.186215:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914048 00000100:00100000:2.0:1769074780.186230:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914048 00000100:00100000:2.0:1769074780.186423:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8677 00000100:00100000:2.0:1769074780.186434:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616914048:12345-192.168.204.60@tcp:4:mirror_io.0 00010000:00010000:2.0:1769074780.186471:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.186489:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.186496:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074780.186498:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074780.186501:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.186639:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.186658:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 10 pid: 36516 timeout: 1868 lvb_type: 0 00000100:00100000:2.0:1769074780.186671:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0001_UUID:28445:1855007625173248:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074780.186762:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074780.190279:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0001_UUID:28445:1855007625173248:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074780.190307:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074780.191682:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.192355:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914176 00000100:00100000:3.0:1769074780.192367:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914176 00010000:00010000:3.0:1769074780.192409:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 11 pid: 36516 timeout: 1868 lvb_type: 0 00000100:00100000:3.0:1769074780.192454:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7137 00000100:00100000:3.0:1769074780.192462:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616914176:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074780.192475:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074780.192493:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 11 pid: 36516 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.192506:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 11 pid: 36516 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.192535:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd522a6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ace8 expref: 11 pid: 36516 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.192553:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.192554:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.192560:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.192576:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.192658:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074780.192714:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:2408:x1855007616914176:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 253us (1039us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.192722:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7137 00010000:00010000:2.0:1769074780.196023:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.196044:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074780.208127:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874546 is committed 00002000:00010000:2.0:1769074780.208417:0:28445:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4808704 00002000:00010000:2.0:1769074780.208423:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074755 -> 1769074779 00002000:00010000:2.0:1769074780.208425:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074755 -> 1769074779 00010000:00010000:2.0:1769074780.208439:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.208452:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.208460:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.208468:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b546117000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074780.208473:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.208481:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.208500:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.208507:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b546117000/0x7fe2323adfd54281 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074780.208731:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2408:x1855007616914048:12345-192.168.204.60@tcp:4:mirror_io.0 Request processed in 22293us (25213us total) trans 17179874546 rc 0/0 00000100:00100000:2.0:1769074780.208743:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8677 00000100:00100000:1.0:1769074780.228415:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.228516:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914432 00000100:00100000:3.0:1769074780.228526:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914432 00000100:00100000:3.0:1769074780.228614:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13345 00000100:00100000:3.0:1769074780.228624:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616914432:12345-192.168.204.60@tcp:36:mirror_io.0 00010000:00010000:3.0:1769074780.228681:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074780.228711:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fc00/0x7fe2323adfd541bd lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c9 expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.228731:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.228736:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074780.229036:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.229057:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.229067:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000001:00080000:3.0:1769074780.229111:0:10283:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589943990 00000004:00010000:3.0:1769074780.229134:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000322f2ffb reply state 000000009ae6a0a6 transno 8589943991 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.229148:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.229153:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.229156:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074780.229158:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074780.229265:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616914432:12345-192.168.204.60@tcp:36:mirror_io.0 Request processed in 641us (860us total) trans 8589943991 rc 0/0 00000100:00100000:3.0:1769074780.229273:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13345 00000020:00080000:3.0:1769074780.229278:0:10283:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00080000:0.0:1769074780.230007:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074780.234895:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.235016:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914560 00000100:00100000:2.0:1769074780.235028:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914560 00000100:00100000:2.0:1769074780.235098:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14251 00000100:00100000:2.0:1769074780.235108:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616914560:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074780.235126:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.235147:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd5428f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c108 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074780.235171:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2b9:0x0] initial LVB size: 2097152, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0xfe8 00010000:00010000:2.0:1769074780.235211:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.235214:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.235222:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd5428f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c108 expref: 10 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.235238:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd5428f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c108 expref: 10 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.235250:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b546117000, rc 0) 00000100:00100000:2.0:1769074780.235315:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616914560:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 208us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.235322:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14251 00000100:00080000:0.0:1769074780.235801:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074780.240178:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.240277:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914688 00000100:00100000:3.0:1769074780.240288:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914688 00000100:00100000:3.0:1769074780.240347:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14252 00000100:00100000:3.0:1769074780.240357:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616914688:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074780.240375:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.240398:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45200/0x7fe2323adfd54296 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c10f expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074780.240422:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bd:0x0] initial LVB size: 3145728, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1000 00010000:00010000:3.0:1769074780.240456:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.240458:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.240467:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45200/0x7fe2323adfd54296 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c10f expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.240482:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45200/0x7fe2323adfd54296 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c10f expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.240495:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45200, rc 0) 00000100:00100000:3.0:1769074780.240573:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62993:x1855007616914688:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 217us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.240581:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14252 00000100:00100000:1.0:1769074780.244727:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.244901:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914816 00000100:00100000:2.0:1769074780.244911:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914816 00000100:00100000:2.0:1769074780.244966:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14253 00000100:00100000:2.0:1769074780.244975:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616914816:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074780.244992:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.245024:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5429d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c116 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074780.245045:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c0:0x0] initial LVB size: 3145728, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1000 00010000:00010000:2.0:1769074780.245066:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.245068:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.245077:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5429d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c116 expref: 10 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.245092:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5429d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c116 expref: 10 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.245110:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:2.0:1769074780.245173:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616914816:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 197us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.245190:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14253 00000100:00100000:1.0:1769074780.248467:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.248559:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616914944 00000100:00100000:3.0:1769074780.248570:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616914944 00000100:00100000:3.0:1769074780.248626:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14254 00000100:00100000:3.0:1769074780.248636:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62993:x1855007616914944:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074780.248656:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.248680:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd542a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c11d expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074780.248703:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2be:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x800 00010000:00010000:3.0:1769074780.248723:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.248726:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.248735:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd542a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c11d expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.248750:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd542a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c11d expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.248761:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba45600, rc 0) 00000100:00100000:3.0:1769074780.248859:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616914944:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 224us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.248866:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14254 00000100:00100000:0.0:1769074780.252400:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.252505:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915072 00000100:00100000:2.0:1769074780.252515:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915072 00000100:00100000:2.0:1769074780.252580:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14255 00000100:00100000:2.0:1769074780.252590:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+10:62993:x1855007616915072:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074780.252607:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.252628:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd542ab lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c124 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074780.252654:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c1:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x800 00010000:00010000:2.0:1769074780.252677:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.252679:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.252688:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd542ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c124 expref: 11 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.252704:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd542ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c124 expref: 11 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.252715:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866800, rc 0) 00000100:00100000:2.0:1769074780.252786:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:62993:x1855007616915072:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 198us (396us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.252793:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14255 00000100:00100000:1.0:1769074780.256240:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.256869:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915200 00000100:00100000:3.0:1769074780.256881:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915200 00000100:00100000:3.0:1769074780.256948:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14256 00000100:00100000:3.0:1769074780.256961:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616915200:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074780.256985:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.257304:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd542b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c12b expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074780.257333:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bf:0x0] initial LVB size: 4808704, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x4b0 00010000:00010000:0.0:1769074780.257356:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.257358:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.257368:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd542b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c12b expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.257382:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd542b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c12b expref: 13 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.257394:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877000, rc 0) 00000100:00100000:0.0:1769074780.257463:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62993:x1855007616915200:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 504us (1236us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.257470:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14256 00000100:00100000:1.0:1769074780.260448:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.260539:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915328 00000100:00100000:2.0:1769074780.260549:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915328 00002000:00010000:2.0:1769074780.260592:0:28445:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/703 (4805728->18446744073709551615) 00002000:00100000:2.0:1769074780.260598:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0df80 with x1855007616915328 ext(4805728->18446744073709551615) 00002000:00010000:2.0:1769074780.260613:0:28445:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0df80. 00000100:00100000:2.0:1769074780.260651:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8678 00000100:00100000:2.0:1769074780.260662:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2408:x1855007616915328:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:2.0:1769074780.261543:0:28445:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4808704 -> 4805728 00002000:00010000:2.0:1769074780.261551:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.261554:0:28445:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.261556:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.261559:0:28445:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:2.0:1769074780.261626:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616915328:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 963us (1185us total) trans 21474841222 rc 0/0 00000100:00100000:2.0:1769074780.261636:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8678 00002000:00010000:2.0:1769074780.261641:0:28445:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/703 (4805728->18446744073709551615) 00002000:00100000:2.0:1769074780.261643:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0df80 with x1855007616915328 ext(4805728->18446744073709551615) 00002000:00010000:2.0:1769074780.261649:0:28445:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b67ff0df80. 00000100:00100000:1.0:1769074780.265448:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.265701:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915456 00000100:00100000:2.0:1769074780.265712:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915456 00002000:00010000:2.0:1769074780.265757:0:28445:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/706 (4194304->18446744073709551615) 00002000:00100000:2.0:1769074780.265763:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0dc00 with x1855007616915456 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1769074780.265899:0:28445:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0dc00. 00000100:00100000:2.0:1769074780.265929:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8679 00000100:00100000:2.0:1769074780.265939:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2408:x1855007616915456:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:2.0:1769074780.266142:0:28445:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.266146:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074755 -> 1769074779 00002000:00010000:2.0:1769074780.266148:0:28445:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.266150:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074755 -> 1769074779 00002000:00010000:2.0:1769074780.266152:0:28445:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074780.266414:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:2408:x1855007616915456:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 475us (975us total) trans 17179874547 rc 0/0 00000100:00100000:2.0:1769074780.266422:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8679 00002000:00010000:2.0:1769074780.266427:0:28445:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/706 (4194304->18446744073709551615) 00002000:00100000:2.0:1769074780.266430:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0dc00 with x1855007616915456 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1769074780.266435:0:28445:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67ff0dc00. 00000100:00100000:1.0:1769074780.270110:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.270240:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915584 00000100:00100000:2.0:1769074780.270252:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915584 00000100:00100000:2.0:1769074780.270316:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13346 00000100:00100000:2.0:1769074780.270327:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616915584:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074780.270353:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.270375:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd542b9 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c132 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.270487:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd542c0 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074780.270504:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd542c0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074780.270516:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd542c0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074780.270713:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:2.0:1769074780.270719:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074780.270744:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd542c0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074780.270894:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd542b9 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c132 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.270916:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd542c0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c132 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.270927:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074780.270930:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074780.270936:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:2.0:1769074780.271125:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62993:x1855007616915584:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 799us (1029us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.271133:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13346 00000100:00100000:1.0:1769074780.276232:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.276367:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915840 00000100:00100000:3.0:1769074780.276378:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915840 00000100:00100000:3.0:1769074780.276547:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13347 00000100:00100000:3.0:1769074780.276559:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62993:x1855007616915840:12345-192.168.204.60@tcp:36:mirror_io.0 00010000:00010000:3.0:1769074780.276618:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074780.276649:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd542c0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c132 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.276672:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.276678:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074780.276917:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074780.276932:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074780.276942:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000001:00080000:3.0:1769074780.276994:0:6512:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589943992 00000004:00010000:3.0:1769074780.277033:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000c0b389ce reply state 000000004e3f73af transno 8589943993 ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074780.277046:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074780.277053:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.277055:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074780.277059:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074780.277160:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62993:x1855007616915840:12345-192.168.204.60@tcp:36:mirror_io.0 Request processed in 604us (938us total) trans 8589943993 rc 0/0 00000100:00100000:3.0:1769074780.277168:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13347 00000020:00080000:3.0:1769074780.277174:0:6512:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1769074780.280315:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.280419:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616915968 00000100:00100000:2.0:1769074780.280429:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616915968 00000100:00100000:2.0:1769074780.280535:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14257 00000100:00100000:2.0:1769074780.280545:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+12:62993:x1855007616915968:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074780.280563:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.280585:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd542ce lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c139 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074780.280607:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bd:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1fe8 00010000:00010000:2.0:1769074780.280629:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.280632:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.280642:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd542ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c139 expref: 13 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.280658:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd542ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c139 expref: 13 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.280669:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866a00, rc 0) 00000100:00100000:2.0:1769074780.280732:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62993:x1855007616915968:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 188us (426us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.280739:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14257 00000100:00100000:1.0:1769074780.283596:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.283738:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916096 00000100:00100000:2.0:1769074780.283750:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916096 00000100:00100000:2.0:1769074780.283853:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14258 00000100:00100000:2.0:1769074780.283864:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:62993:x1855007616916096:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:2.0:1769074780.283881:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.283905:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd542d5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c140 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074780.283928:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c3:0x0] initial LVB size: 8388608, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x2000 00010000:00010000:2.0:1769074780.283949:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.283951:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.283960:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd542d5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c140 expref: 14 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.283976:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd542d5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c140 expref: 14 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.283994:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867c00, rc 0) 00000100:00100000:2.0:1769074780.284056:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62993:x1855007616916096:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 195us (471us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.284063:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14258 00000100:00100000:1.0:1769074780.287785:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.288087:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916224 00000100:00100000:0.0:1769074780.288100:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916224 00000100:00100000:0.0:1769074780.288166:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14259 00000100:00100000:0.0:1769074780.288177:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:62993:x1855007616916224:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:0.0:1769074780.288204:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.288227:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd542dc lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c147 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074780.288259:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c4:0x0] initial LVB size: 4808704, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x4b0 00010000:00010000:0.0:1769074780.288283:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.288286:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.288294:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd542dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c147 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.288311:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd542dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c147 expref: 15 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.288324:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876600, rc 0) 00000100:00100000:0.0:1769074780.288394:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62993:x1855007616916224:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 218us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.288402:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14259 00000100:00100000:1.0:1769074780.291474:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.291579:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916352 00000100:00100000:2.0:1769074780.291591:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916352 00002000:00010000:2.0:1769074780.291642:0:28445:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/708 (4805728->18446744073709551615) 00002000:00100000:2.0:1769074780.291647:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6d80 with x1855007616916352 ext(4805728->18446744073709551615) 00002000:00010000:2.0:1769074780.291662:0:28445:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6d80. 00000100:00100000:2.0:1769074780.291701:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8680 00000100:00100000:2.0:1769074780.291712:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616916352:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:2.0:1769074780.292128:0:28445:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4808704 -> 4805728 00002000:00010000:2.0:1769074780.292133:0:28445:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.292135:0:28445:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.292138:0:28445:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.292140:0:28445:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:2.0:1769074780.292211:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616916352:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 501us (749us total) trans 17179874548 rc 0/0 00000100:00100000:2.0:1769074780.292219:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8680 00002000:00010000:2.0:1769074780.292222:0:28445:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/708 (4805728->18446744073709551615) 00002000:00100000:2.0:1769074780.292225:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6d80 with x1855007616916352 ext(4805728->18446744073709551615) 00002000:00010000:2.0:1769074780.292230:0:28445:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6d80. 00000100:00100000:1.0:1769074780.295559:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.295721:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916480 00000100:00100000:0.0:1769074780.295733:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916480 00002000:00010000:0.0:1769074780.295792:0:8403:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/704 (4194304->18446744073709551615) 00002000:00100000:0.0:1769074780.295799:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7480 with x1855007616916480 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1769074780.295860:0:8403:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d7480. 00000100:00100000:0.0:1769074780.295901:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8681 00000100:00100000:0.0:1769074780.295912:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+14:2408:x1855007616916480:12345-192.168.204.60@tcp:10:mirror_io.0 00002000:00010000:0.0:1769074780.296095:0:8403:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074780.296098:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074755 -> 1769074779 00002000:00010000:0.0:1769074780.296101:0:8403:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.296103:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074755 -> 1769074779 00002000:00010000:0.0:1769074780.296106:0:8403:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074780.296160:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616916480:12345-192.168.204.60@tcp:10:mirror_io.0 Request processed in 250us (611us total) trans 21474841223 rc 0/0 00000100:00100000:0.0:1769074780.296168:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8681 00002000:00010000:0.0:1769074780.296171:0:8403:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/704 (4194304->18446744073709551615) 00002000:00100000:0.0:1769074780.296174:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d7480 with x1855007616916480 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1769074780.296178:0:8403:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5436d7480. 00000100:00100000:1.0:1769074780.300281:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.300408:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916736 00000100:00100000:2.0:1769074780.300420:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916736 00000100:00100000:2.0:1769074780.300513:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4690 00000100:00100000:2.0:1769074780.300524:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:62993:x1855007616916736:12345-192.168.204.60@tcp:35:mirror_io.0 00000004:00010000:2.0:1769074780.300613:0:6515:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x200000405:0x1fb:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd54169 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0b4 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.300676:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074780.300691:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.300700:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074780.300741:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.300756:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 15 pid: 6514 timeout: 1868 lvb_type: 0 00000100:00100000:2.0:1769074780.300771:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625173376:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074780.300916:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074780.304969:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d0380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625173376:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074780.305020:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:1.0:1769074780.307310:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.307412:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916864 00000100:00100000:0.0:1769074780.307424:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916864 00010000:00010000:0.0:1769074780.307471:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 16 pid: 6514 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.307510:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7138 00000100:00100000:0.0:1769074780.307520:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2408:x1855007616916864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074780.307546:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074780.307553:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 16 pid: 6514 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.307566:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 16 pid: 6514 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.307589:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd541af lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0c2 expref: 16 pid: 6514 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.307608:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.307614:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074780.307623:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074780.307632:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074780.307669:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074780.307741:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2408:x1855007616916864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 222us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.307750:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7138 00010000:00010000:2.0:1769074780.308123:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074780.308134:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000040:00080000:2.0:1769074780.308442:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2420, 88 off207176 00000004:00080000:2.0:1769074780.308452:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150644376ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308498:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2419, 88 off206824 00000004:00080000:2.0:1769074780.308504:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150696971ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308521:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2420, 88 off206912 00000004:00080000:2.0:1769074780.308536:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150728673ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308548:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2421, 88 off207264 00000004:00080000:2.0:1769074780.308551:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150744570ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308561:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2422, 88 off207352 00000004:00080000:2.0:1769074780.308564:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150757491ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308574:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2421, 88 off207000 00000004:00080000:2.0:1769074780.308578:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150770706ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308587:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2422, 88 off207088 00000004:00080000:2.0:1769074780.308591:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150783806ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308599:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2423, 88 off207176 00000004:00080000:2.0:1769074780.308602:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150795190ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308610:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2423, 88 off207440 00000004:00080000:2.0:1769074780.308613:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150805883ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308621:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2424, 88 off207264 00000004:00080000:2.0:1769074780.308625:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150817889ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308634:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2424, 88 off207528 00000004:00080000:2.0:1769074780.308638:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150831416ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308648:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2425, 88 off207352 00000004:00080000:2.0:1769074780.308652:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150844828ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074780.308661:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2425, 88 off207616 00000004:00080000:2.0:1769074780.308664:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150857404ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308673:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2426, 88 off207704 00000004:00080000:2.0:1769074780.308676:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150868978ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308685:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2427, 88 off207792 00000004:00080000:2.0:1769074780.308688:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150881401ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308697:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2428, 88 off207880 00000004:00080000:2.0:1769074780.308701:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768150894096ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074780.308711:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2426, 88 off207440 00000004:00080000:2.0:1769074780.308714:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768150907511ns, next at 1706953247299ns, rc = 0 00000001:00080000:0.0:1769074780.312015:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943994 is committed 00000004:00080000:0.0:1769074780.312028:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1930 committed 00000004:00080000:0.0:1769074780.312035:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1931 committed 00000004:00080000:0.0:1769074780.312041:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1930 committed 00000004:00080000:0.0:1769074780.312043:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1929 committed 00000004:00080000:0.0:1769074780.312046:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1928 committed 00000004:00080000:0.0:1769074780.312048:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1929 committed 00000004:00080000:0.0:1769074780.312049:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1927 committed 00000004:00080000:0.0:1769074780.312051:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1928 committed 00000004:00080000:0.0:1769074780.312053:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1926 committed 00000004:00080000:0.0:1769074780.312055:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1927 committed 00000004:00080000:0.0:1769074780.312057:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1926 committed 00000004:00080000:0.0:1769074780.312059:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1925 committed 00000004:00080000:0.0:1769074780.312061:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1925 committed 00000004:00080000:0.0:1769074780.312062:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1924 committed 00000004:00080000:0.0:1769074780.312064:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1924 committed 00000004:00080000:0.0:1769074780.312066:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1923 committed 00000004:00080000:0.0:1769074780.312068:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1923 committed 00010000:00010000:0.0:1769074780.312098:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312110:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312120:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312126:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074780.312130:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312137:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312152:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.312154:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074780.312159:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312164:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54288 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074780.312194:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.312200:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.312206:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.312211:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074780.312214:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.312220:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.312231:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.312234:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074780.312236:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.312241:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd542c7 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:2.0:1769074780.312618:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625173888:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.312692:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.312740:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625174400:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.312760:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.312796:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1769074780.312803:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625173760:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.312847:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.312866:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625174272:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.312883:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.312916:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625173888 00000100:00100000:2.0:1769074780.312924:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625173888 00000100:00100000:2.0:1769074780.312987:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174400 00000100:00100000:2.0:1769074780.312991:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174400 00000100:00100000:2.0:1769074780.313130:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625173760 00000100:00100000:2.0:1769074780.313134:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625173760 00000100:00100000:2.0:1769074780.313149:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174272 00000100:00100000:2.0:1769074780.313152:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174272 00000100:00100000:2.0:1769074780.313172:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14260 00000100:00100000:2.0:1769074780.313194:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3650:x1855007625173888:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.313370:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:2.0:1769074780.313374:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.313376:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.313378:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.313381:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:2.0:1769074780.313425:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3650:x1855007625173888:12345-0@lo:2:osp-syn-1-0.0 Request processed in 244us (732us total) trans 17179874549 rc 0/0 00000100:00100000:2.0:1769074780.313432:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14260 00000100:00100000:2.0:1769074780.313454:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14261 00000100:00100000:2.0:1769074780.313461:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3650:x1855007625174400:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.313514:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb size from disk: 8388608 -> 8388608 00002000:00010000:2.0:1769074780.313517:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.313520:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.313521:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.313523:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb blocks from disk: 8192 -> 8192 00000100:00100000:2.0:1769074780.313547:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3650:x1855007625174400:12345-0@lo:2:osp-syn-1-0.0 Request processed in 86us (785us total) trans 17179874550 rc 0/0 00000100:00100000:2.0:1769074780.313553:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14261 00000100:00100000:2.0:1769074780.313566:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14262 00000100:00100000:2.0:1769074780.313572:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3650:x1855007625173760:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.313620:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:2.0:1769074780.313623:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074777 -> 1769074777 00002000:00010000:2.0:1769074780.313626:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime from disk: 1769074777 -> 1769074777 00002000:00010000:2.0:1769074780.313628:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074777 -> 1769074777 00002000:00010000:2.0:1769074780.313630:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb blocks from disk: 5296 -> 5296 00000100:00100000:2.0:1769074780.313654:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3650:x1855007625173760:12345-0@lo:2:osp-syn-1-0.0 Request processed in 83us (806us total) trans 17179874551 rc 0/0 00000100:00100000:2.0:1769074780.313661:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14262 00000100:00100000:2.0:1769074780.313672:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14263 00000100:00100000:2.0:1769074780.313678:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3650:x1855007625174272:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.313724:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.313727:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.313730:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.313732:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.313734:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:2.0:1769074780.313757:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3650:x1855007625174272:12345-0@lo:2:osp-syn-1-0.0 Request processed in 79us (872us total) trans 17179874552 rc 0/0 00000100:00100000:2.0:1769074780.313764:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14263 00000100:00080000:2.0:1769074780.314414:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074780.314678:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625174912:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.314726:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074780.314868:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625173632:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074780.314903:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625174784:0@lo:2:osp-syn-0-0.0 00000004:00010000:3.0:1769074780.314993:0:6515:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000febab4b9 reply state 00000000cc3177e0 transno 8589943994 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315029:0:6515:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315040:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.315043:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:3.0:1769074780.315048:0:6515:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1769074780.315053:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.315056:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074780.315062:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0f600/0x7fe2323adfd54169 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0b4 expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00080000:3.0:1769074780.315189:0:6515:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1769074780.315206:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62993:x1855007616916736:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 14684us (14940us total) trans 8589943994 rc 0/0 00000100:00100000:3.0:1769074780.315215:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4690 00010000:00010000:3.0:1769074780.315282:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315291:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315304:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315309:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074780.315314:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315323:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315333:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.315335:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074780.315338:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074780.315344:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd542e3 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00000100:00100000:2.0:1769074780.315642:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625175424:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.315672:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.315958:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625173504:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.319109:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874539 gen 2 00000004:00080000:2.0:1769074780.319124:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d4a80, transno 17179874531 00000004:00080000:2.0:1769074780.319135:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d5f80, transno 17179874532 00000004:00080000:2.0:1769074780.319139:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d7b80, transno 17179874533 00000004:00080000:2.0:1769074780.319141:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d5c00, transno 17179874534 00000004:00080000:2.0:1769074780.319144:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d4380, transno 17179874535 00000004:00080000:2.0:1769074780.319146:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545389880, transno 17179874536 00000004:00080000:2.0:1769074780.319149:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d5180, transno 17179874537 00000004:00080000:2.0:1769074780.319151:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5436d4700, transno 17179874538 00000004:00080000:2.0:1769074780.319154:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f93b80, transno 17179874539 00000100:00100000:2.0:1769074780.319163:0:3650:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b54cf6b100 x1855007625173760/t17179874551(17179874551) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074796 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1769074780.319175:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf6b100/2, rc 0, transno 17179874551 00000100:00100000:2.0:1769074780.319191:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625173760:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.319206:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf68380/2, rc 0, transno 17179874552 00000100:00100000:2.0:1769074780.319210:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625174272:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.319219:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf69880/2, rc 0, transno 17179874549 00000100:00100000:2.0:1769074780.319222:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625173888:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.319231:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460eed80/2, rc 0, transno 17179874550 00000100:00100000:2.0:1769074780.319234:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625174400:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.319286:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074780.319487:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174912 00000100:00100000:0.0:1769074780.319495:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174912 00000100:00100000:0.0:1769074780.319563:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625175424 00000100:00100000:0.0:1769074780.319567:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625175424 00000100:00100000:0.0:1769074780.319592:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14264 00000100:00100000:0.0:1769074780.319601:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625174912:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.319798:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074780.319803:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.319820:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.319822:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.319824:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:0.0:1769074780.319888:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625174912:12345-0@lo:2:osp-syn-0-0.0 Request processed in 288us (5160us total) trans 21474841224 rc 0/0 00000100:00100000:0.0:1769074780.319895:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14264 00000100:00100000:0.0:1769074780.319913:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14265 00000100:00100000:0.0:1769074780.319920:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625175424:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.319972:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074780.319975:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.319977:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.319980:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.319982:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074780.320020:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625175424:12345-0@lo:2:osp-syn-0-0.0 Request processed in 101us (4347us total) trans 21474841225 rc 0/0 00000100:00100000:0.0:1769074780.320026:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14265 00000100:00080000:3.0:1769074780.320792:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:3.0:1769074780.320955:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 8 in progress, 4 in flight, 9 done 00000100:00100000:3.0:1769074780.321216:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841215 gen 2 00000004:00080000:3.0:1769074780.321221:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f93800, transno 21474841208 00000004:00080000:3.0:1769074780.321230:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f91c00, transno 21474841209 00000004:00080000:3.0:1769074780.321232:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f93100, transno 21474841210 00000004:00080000:3.0:1769074780.321235:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f91180, transno 21474841211 00000004:00080000:3.0:1769074780.321238:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f92300, transno 21474841212 00000004:00080000:3.0:1769074780.321240:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f90e00, transno 21474841213 00000004:00080000:3.0:1769074780.321243:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f90000, transno 21474841214 00000004:00080000:3.0:1769074780.321246:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f92a00, transno 21474841215 00000100:00100000:3.0:1769074780.321252:0:3650:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b543858e00 x1855007625174912/t21474841224(21474841224) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074796 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:3.0:1769074780.321263:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b543858e00/2, rc 0, transno 21474841224 00000100:00100000:3.0:1769074780.321267:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625174912:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1769074780.321279:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5476fa680/2, rc 0, transno 21474841225 00000100:00100000:3.0:1769074780.321283:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625175424:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1769074780.321362:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 8 in progress, 6 in flight, 8 done 00000100:00100000:3.0:1769074780.321433:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.321461:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616916992 00000100:00100000:3.0:1769074780.321467:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616916992 00000100:00100000:3.0:1769074780.321506:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13348 00000100:00100000:3.0:1769074780.321515:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:62993:x1855007616916992:12345-192.168.204.60@tcp:101:mirror_io.0 00010000:00010000:3.0:1769074780.321637:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.321662:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c14e expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.321724:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.321738:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.321761:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.321765:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074780.321770:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00000100:00100000:3.0:1769074780.321928:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62993:x1855007616916992:12345-192.168.204.60@tcp:101:mirror_io.0 Request processed in 414us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.321936:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13348 00000100:00100000:2.0:1769074780.326478:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.326512:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625174016:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.326578:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.326598:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625174528:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.326618:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.326636:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625175040:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.326655:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.326683:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625175552:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.326703:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.326737:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625173632 00000100:00100000:2.0:1769074780.326746:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625173632 00000100:00100000:2.0:1769074780.326802:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625173504 00000100:00100000:2.0:1769074780.326830:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625173504 00000100:00100000:2.0:1769074780.326851:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14266 00000100:00100000:2.0:1769074780.326859:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625173632:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074780.327207:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174016 00000100:00100000:0.0:1769074780.327212:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174016 00000100:00100000:0.0:1769074780.327238:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174528 00000100:00100000:0.0:1769074780.327241:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174528 00000100:00100000:0.0:1769074780.327265:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625175040 00000100:00100000:0.0:1769074780.327268:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625175040 00000100:00100000:0.0:1769074780.327279:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625175552 00000100:00100000:0.0:1769074780.327281:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625175552 00000100:00100000:0.0:1769074780.327292:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14267 00000100:00100000:0.0:1769074780.327299:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625173504:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074780.327392:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:0.0:1769074780.327396:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074772 -> 1769074772 00002000:00010000:0.0:1769074780.327398:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime from disk: 1769074770 -> 1769074770 00002000:00010000:0.0:1769074780.327400:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074772 -> 1769074772 00002000:00010000:0.0:1769074780.327402:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb blocks from disk: 2024 -> 2024 00000100:00100000:0.0:1769074780.327450:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625173504:12345-0@lo:2:osp-syn-1-0.0 Request processed in 151us (980us total) trans 17179874554 rc 0/0 00000100:00100000:0.0:1769074780.327456:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14267 00000100:00100000:0.0:1769074780.327473:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14268 00000100:00100000:0.0:1769074780.327479:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625174016:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074780.327528:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074780.327532:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.327534:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.327536:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.327538:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074780.327563:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625174016:12345-0@lo:2:osp-syn-1-0.0 Request processed in 84us (985us total) trans 17179874555 rc 0/0 00000100:00100000:0.0:1769074780.327570:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14268 00000100:00100000:0.0:1769074780.327583:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14269 00000100:00100000:0.0:1769074780.327589:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625174528:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.327650:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074780.327653:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.327655:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.327657:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.327658:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074780.327683:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625174528:12345-0@lo:2:osp-syn-0-0.0 Request processed in 95us (1064us total) trans 21474841226 rc 0/0 00000100:00100000:0.0:1769074780.327689:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14269 00000100:00100000:0.0:1769074780.327701:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14270 00000100:00100000:0.0:1769074780.327707:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625175040:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.327752:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:0.0:1769074780.327755:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.327757:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.327759:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.327762:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074780.327800:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625175040:12345-0@lo:2:osp-syn-0-0.0 Request processed in 92us (1143us total) trans 21474841227 rc 0/0 00000100:00100000:0.0:1769074780.327835:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14270 00000100:00100000:0.0:1769074780.327849:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14271 00000100:00100000:0.0:1769074780.327857:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+13:3651:x1855007625175552:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074780.327909:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074780.327912:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.327914:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.327915:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.327917:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074780.327943:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625175552:12345-0@lo:2:osp-syn-1-0.0 Request processed in 88us (1239us total) trans 17179874556 rc 0/0 00000100:00100000:0.0:1769074780.327949:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14271 00000100:00100000:0.0:1769074780.327972:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074780.328016:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625175296:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074780.328043:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074780.328067:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174784 00000100:00100000:0.0:1769074780.328071:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174784 00000100:00100000:0.0:1769074780.328086:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625175296 00000100:00100000:0.0:1769074780.328088:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625175296 00000100:00100000:0.0:1769074780.328098:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14272 00000100:00100000:0.0:1769074780.328104:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+10:3649:x1855007625174784:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.328154:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074780.328157:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074777 -> 1769074777 00002000:00010000:0.0:1769074780.328159:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.328161:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074777 -> 1769074777 00002000:00010000:0.0:1769074780.328163:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:0.0:1769074780.328201:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625174784:12345-0@lo:2:osp-syn-0-0.0 Request processed in 97us (228us total) trans 21474841228 rc 0/0 00000100:00100000:0.0:1769074780.328207:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14272 00000100:00100000:0.0:1769074780.328220:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14273 00000100:00100000:0.0:1769074780.328226:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+10:3649:x1855007625175296:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.328273:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074780.328276:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.328277:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.328279:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.328282:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074780.328306:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625175296:12345-0@lo:2:osp-syn-0-0.0 Request processed in 80us (262us total) trans 21474841229 rc 0/0 00000100:00100000:0.0:1769074780.328312:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14273 00000004:00080000:0.0:1769074780.328355:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54385ad80/2, rc 0, transno 21474841228 00000100:00100000:0.0:1769074780.328363:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625174784:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074780.328373:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5476f8e00/2, rc 0, transno 21474841229 00000100:00100000:0.0:1769074780.328376:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625175296:0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074780.328714:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074780.328718:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.328720:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.328722:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.328724:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:2.0:1769074780.328757:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625173632:12345-0@lo:2:osp-syn-1-0.0 Request processed in 1899us (9471us total) trans 17179874553 rc 0/0 00000100:00100000:2.0:1769074780.328765:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14266 00000004:00080000:2.0:1769074780.328848:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54510bb80/2, rc 0, transno 17179874556 00000100:00100000:2.0:1769074780.328853:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625175552:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.328863:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf68a80/2, rc 0, transno 17179874554 00000100:00100000:2.0:1769074780.328867:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625173504:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.328876:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf69c00/2, rc 0, transno 17179874555 00000100:00100000:2.0:1769074780.328879:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625174016:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.328887:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54385aa00/2, rc 0, transno 21474841226 00000100:00100000:2.0:1769074780.328891:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625174528:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074780.328900:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5476f8380/2, rc 0, transno 21474841227 00000100:00100000:2.0:1769074780.328903:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625175040:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074780.328938:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf68000/2, rc 0, transno 17179874553 00000100:00100000:2.0:1769074780.328941:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625173632:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.328950:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625174144:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.328978:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.329140:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625174656:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.329166:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.329200:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174144 00000100:00100000:2.0:1769074780.329205:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174144 00000100:00100000:2.0:1769074780.329233:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625174656 00000100:00100000:2.0:1769074780.329236:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625174656 00000100:00100000:2.0:1769074780.329253:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14274 00000100:00100000:2.0:1769074780.329260:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625174144:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.329359:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.329362:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.329365:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.329367:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.329369:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074780.329395:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625174144:12345-0@lo:2:osp-syn-1-0.0 Request processed in 135us (416us total) trans 17179874557 rc 0/0 00000100:00100000:2.0:1769074780.329402:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14274 00000100:00100000:2.0:1769074780.329414:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14275 00000100:00100000:2.0:1769074780.329421:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625174656:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074780.329482:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074780.329485:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.329487:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.329490:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074775 -> 1769074775 00002000:00010000:2.0:1769074780.329491:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074780.329514:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625174656:12345-0@lo:2:osp-syn-0-0.0 Request processed in 94us (347us total) trans 21474841230 rc 0/0 00000100:00100000:2.0:1769074780.329521:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14275 00000004:00080000:2.0:1769074780.329765:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b543859880/2, rc 0, transno 21474841230 00000100:00100000:2.0:1769074780.329867:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625174656:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.329877:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625175168:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.329905:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.329926:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625175168 00000100:00100000:2.0:1769074780.329931:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625175168 00000100:00100000:2.0:1769074780.329953:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14276 00000100:00100000:2.0:1769074780.329959:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625175168:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074780.330222:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.330225:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.330227:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.330229:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.330231:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074780.330258:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625175168:12345-0@lo:2:osp-syn-0-0.0 Request processed in 299us (353us total) trans 21474841231 rc 0/0 00000100:00100000:2.0:1769074780.330264:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14276 00000004:00080000:2.0:1769074780.330292:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5476fb480/2, rc 0, transno 21474841231 00000100:00100000:2.0:1769074780.330297:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625175168:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074780.330318:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54cf69180/2, rc 0, transno 17179874557 00000100:00100000:2.0:1769074780.330322:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625174144:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.332481:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.332629:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616917248 00000100:00100000:3.0:1769074780.332639:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616917248 00000100:00100000:3.0:1769074780.332731:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4691 00000100:00100000:3.0:1769074780.332742:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:62993:x1855007616917248:12345-192.168.204.60@tcp:35:mirror_io.0 00000100:00100000:3.0:1769074780.333048:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:62993:x1855007616917248:12345-192.168.204.60@tcp:35:mirror_io.0 Request processed in 307us (578us total) trans 8589943995 rc 0/0 00000100:00100000:3.0:1769074780.333056:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4691 00000100:00100000:0.0:1769074780.379546:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.379847:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616917504 00000100:00100000:3.0:1769074780.379859:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616917504 00000100:00100000:3.0:1769074780.379945:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13349 00000100:00100000:3.0:1769074780.379955:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+220:63001:x1855007616917504:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074780.379980:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.380007:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd542f1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c155 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.380088:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380104:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380116:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380132:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380140:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380146:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380272:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54306 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380282:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54306 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380289:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54306 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074780.380298:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:3.0:1769074780.380376:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380384:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380389:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380395:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380400:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602b800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074780.380404:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380412:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380423:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074780.380426:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074780.380430:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380438:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b800/0x7fe2323adfd542f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380452:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380457:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380463:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380468:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074780.380470:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380475:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380481:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b681690300) 00010000:00010000:3.0:1769074780.380484:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074780.380486:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380491:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd542ff lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074780.380549:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd542f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c155 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.380562:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54306 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c155 expref: 222 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.380572:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.380574:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074780.380577:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88400, rc 0) 00000100:00100000:3.0:1769074780.380657:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63001:x1855007616917504:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 703us (1122us total) trans 8589943996 rc 0/0 00000100:00100000:3.0:1769074780.380665:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13349 00000100:00080000:2.0:1769074780.380862:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074780.382693:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.382797:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616917760 00000100:00100000:0.0:1769074780.382838:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616917760 00000100:00100000:0.0:1769074780.382901:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13350 00000100:00100000:0.0:1769074780.382910:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+7:63002:x1855007616917760:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:0.0:1769074780.382933:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.382953:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd54314 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c15c expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.383031:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383046:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383057:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383070:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383076:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383082:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383206:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383216:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383223:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074780.383232:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:0.0:1769074780.383318:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383325:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383332:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383338:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383342:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074780.383347:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383354:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383367:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074780.383370:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074780.383375:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383381:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876a00/0x7fe2323adfd5431b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383394:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383399:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383405:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383410:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074780.383412:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383418:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383424:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b65436ae00) 00010000:00010000:0.0:1769074780.383426:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074780.383429:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383437:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54322 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074780.383475:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd54314 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.383487:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.383494:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.383496:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074780.383499:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876c00, rc 0) 00000100:00080000:2.0:1769074780.383750:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074780.383911:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+9:63002:x1855007616917760:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 1000us (1225us total) trans 8589943997 rc 0/0 00000100:00100000:0.0:1769074780.383923:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13350 00000100:00100000:3.0:1769074780.396776:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.406434:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616918272 00000100:00100000:2.0:1769074780.406452:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616918272 00000100:00100000:2.0:1769074780.406756:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13351 00000100:00100000:2.0:1769074780.406942:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:63006:x1855007616918272:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074780.406985:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.407399:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866200/0x7fe2323adfd54337 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c16a expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.407947:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074780.407968:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074780.407982:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074780.408119:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:2.0:1769074780.408293:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866200/0x7fe2323adfd54337 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.408318:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.408329:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074780.408332:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1769074780.408337:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867000, rc 0) 00000100:00100000:2.0:1769074780.408437:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+11:63006:x1855007616918272:12345-192.168.204.60@tcp:101:cat.0 Request processed in 1667us (11672us total) trans 8589943998 rc 0/0 00000100:00100000:2.0:1769074780.408447:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13351 00000100:00100000:3.0:1769074780.409270:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.410132:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616918016 00000100:00100000:1.0:1769074780.410144:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616918016 00000100:00100000:1.0:1769074780.410209:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13352 00000100:00100000:1.0:1769074780.410218:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63007:x1855007616918016:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074780.410243:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.410263:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd5434c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c163 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.410357:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074780.410372:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074780.410387:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074780.410397:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:1.0:1769074780.410503:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd5434c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 225 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.410520:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.410541:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074780.410544:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:1.0:1769074780.410547:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fe00, rc 0) 00000100:00100000:1.0:1769074780.410595:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63007:x1855007616918016:12345-192.168.204.60@tcp:101:cat.0 Request processed in 378us (1328us total) trans 8589943999 rc 0/0 00000100:00100000:1.0:1769074780.410602:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13352 00000100:00100000:1.0:1769074780.410644:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.410661:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616918528 00000100:00100000:1.0:1769074780.410665:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616918528 00000100:00100000:1.0:1769074780.410684:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13353 00000100:00100000:1.0:1769074780.410690:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63001:x1855007616918528:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074780.410697:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.410699:0:8855:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074780.410711:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54306 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c155 expref: 224 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.410722:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074780.410726:0:8855:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074780.410731:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54361 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c171 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.410755:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074780.410765:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.411655:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.411668:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.411675:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.411683:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 7/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074780.411718:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.411731:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 224 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:1.0:1769074780.411746:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625175680:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074780.411985:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.412058:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 10 pid: 10283 timeout: 1868 lvb_type: 0 00000100:00100000:1.0:1769074780.412069:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625175808:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074780.412097:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.412105:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 8 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:1.0:1769074780.412113:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625175936:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074780.412133:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.412141:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 10 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:1.0:1769074780.412148:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625176064:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074780.412162:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074780.417671:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.417695:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.417726:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.417750:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.418297:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616918912 00000100:00100000:0.0:1769074780.418312:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616918912 00000100:00100000:0.0:1769074780.418338:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616918656 00000100:00100000:0.0:1769074780.418343:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616918656 00010000:00010000:0.0:1769074780.418384:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 11 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.418426:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616918784 00000100:00100000:0.0:1769074780.418432:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616918784 00010000:00010000:0.0:1769074780.418461:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 226 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.418485:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919040 00000100:00100000:0.0:1769074780.418490:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919040 00010000:00010000:0.0:1769074780.418508:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 12 pid: 10283 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.418558:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7139 00000100:00100000:0.0:1769074780.418568:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616918656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074780.418587:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074780.418593:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 12 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.418605:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 12 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.418628:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.418635:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.418644:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.418651:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074780.418705:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+12:2409:x1855007616918656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 137us (1046us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.418713:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7139 00000100:00100000:0.0:1769074780.418740:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7140 00000100:00100000:0.0:1769074780.418750:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2408:x1855007616918784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074780.418761:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074780.418767:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 226 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.419238:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 226 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.419256:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.419261:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.419269:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.419275:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074780.419312:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2408:x1855007616918784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 563us (1613us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.419320:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7140 00000100:00100000:0.0:1769074780.419343:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7141 00000100:00100000:0.0:1769074780.419353:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616919040:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074780.419364:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074780.419371:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 11 pid: 10283 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.419382:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 11 pid: 10283 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.419395:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.419400:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.419406:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.419413:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074780.419496:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2409:x1855007616919040:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 87us (1686us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.419504:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7141 00000100:00100000:0.0:1769074780.419584:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13354 00000100:00100000:0.0:1769074780.419592:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63007:x1855007616918912:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.419614:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.419630:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5436f lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c178 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.419736:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074780.419746:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.419752:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074780.419769:0:6514:0:(ldlm_request.c:362:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 12 type: IBT flags: 0x40210400000020 pid: 8855 initiator: MDT0 00000100:00100000:3.0:1769074780.420612:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.420652:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919168 00000100:00100000:3.0:1769074780.420661:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919168 00000100:00100000:3.0:1769074780.420693:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13355 00000100:00100000:3.0:1769074780.420702:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+11:63006:x1855007616919168:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074780.420723:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.420748:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5437d lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c17f expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.421151:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074780.421324:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074780.421336:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074780.421347:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000100:00100000:3.0:1769074780.421431:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:0.0:1769074780.421877:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fe00/0x7fe2323adfd54353 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c163 expref: 226 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.421895:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625175680:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074780.421912:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5433e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c16a expref: 12 pid: 10283 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.421922:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625175808:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074780.421933:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625175936:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074780.421943:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd54329 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c15c expref: 11 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.421953:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625176064:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074780.421976:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x40210400000020 pid: 8855 initiator: MDT0 00000100:00100000:0.0:1769074780.423057:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919296 00000100:00100000:0.0:1769074780.423068:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919296 00010000:00010000:0.0:1769074780.423105:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 11 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 9 pid: 6512 timeout: 1868 lvb_type: 0 00000100:00100000:0.0:1769074780.423139:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7142 00000100:00100000:0.0:1769074780.423148:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+9:2407:x1855007616919296:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074780.423162:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074780.423168:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 9 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.423193:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 9 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.423218:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd542ea lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 12 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c14e expref: 9 pid: 6512 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.423235:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.423239:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074780.423248:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x40210400000020 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.423255:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x40290400000020 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.423263:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 11 type: IBT flags: 0x40290400000020 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.423270:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074780.423275:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074780.423297:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074780.423349:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+8:2407:x1855007616919296:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 201us (1919us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.423356:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7142 00010000:00010000:0.0:1769074780.423384:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT flags: 0x40210400000020 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074780.423390:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT flags: 0x40210400000020 pid: 8855 initiator: MDT0 00000040:00080000:0.0:1769074780.424769:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2429, 88 off207968 00000004:00080000:0.0:1769074780.425072:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768267263411ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.425142:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2427, 88 off207528 00000004:00080000:0.0:1769074780.425148:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768267340989ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.425359:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2428, 88 off207616 00000004:00080000:0.0:1769074780.425366:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768267559189ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.426825:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2430, 88 off208056 00000004:00080000:0.0:1769074780.426840:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768269033122ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.426861:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2431, 88 off208144 00000004:00080000:0.0:1769074780.426866:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768269058660ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.426881:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2429, 88 off207704 00000004:00080000:0.0:1769074780.426885:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768269077790ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.427072:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2430, 88 off207792 00000004:00080000:0.0:1769074780.427077:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768269270270ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.427152:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2431, 88 off207880 00000004:00080000:0.0:1769074780.427156:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768269348761ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.427240:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2432, 88 off208232 00000004:00080000:0.0:1769074780.427245:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768269437652ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.427269:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2432, 88 off207968 00000004:00080000:0.0:1769074780.427276:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768269469298ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.427474:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2433, 88 off208320 00000004:00080000:0.0:1769074780.427480:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768269673392ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.427504:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2433, 88 off208056 00000004:00080000:0.0:1769074780.427508:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768269701412ns, next at 1706953247299ns, rc = 0 00000040:00080000:0.0:1769074780.427775:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2434, 88 off208408 00000004:00080000:0.0:1769074780.427781:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768269973818ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.427923:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2435, 88 off208496 00000004:00080000:0.0:1769074780.427927:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768270119963ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.428030:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2436, 88 off208584 00000004:00080000:0.0:1769074780.428037:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768270229918ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.428126:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2437, 88 off208672 00000004:00080000:0.0:1769074780.428131:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1768270323946ns, next at 1706951295843ns, rc = 0 00000040:00080000:0.0:1769074780.428214:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2434, 88 off208144 00000004:00080000:0.0:1769074780.428218:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1768270411059ns, next at 1706953247299ns, rc = 0 00000001:00080000:3.0:1769074780.430259:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944000 is committed 00000004:00080000:3.0:1769074780.430288:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1938 committed 00000004:00080000:3.0:1769074780.430356:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1940 committed 00000004:00080000:3.0:1769074780.430372:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1939 committed 00000004:00080000:3.0:1769074780.430375:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1938 committed 00000004:00080000:3.0:1769074780.430377:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1937 committed 00000004:00080000:3.0:1769074780.430380:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1937 committed 00000100:00100000:2.0:1769074780.430644:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625176576:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.430722:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.430759:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625177088:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.430776:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.430856:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1769074780.430862:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625176192:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.430883:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.430902:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625176704:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.430921:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.430959:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176576 00000100:00100000:2.0:1769074780.430968:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176576 00000100:00100000:2.0:1769074780.431011:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177088 00000100:00100000:2.0:1769074780.431015:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177088 00000100:00100000:2.0:1769074780.431031:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176192 00000100:00100000:2.0:1769074780.431033:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176192 00000100:00100000:2.0:1769074780.431047:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176704 00000100:00100000:2.0:1769074780.431049:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176704 00000100:00100000:2.0:1769074780.431070:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14277 00000100:00100000:2.0:1769074780.431079:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+17:3651:x1855007625176576:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.431264:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:2.0:1769074780.431268:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.431271:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.431273:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.431276:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:2.0:1769074780.431316:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+18:3651:x1855007625176576:12345-0@lo:2:osp-syn-1-0.0 Request processed in 237us (593us total) trans 17179874558 rc 0/0 00000100:00100000:2.0:1769074780.431325:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14277 00000100:00100000:2.0:1769074780.431344:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14278 00000100:00100000:2.0:1769074780.431350:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+17:3651:x1855007625177088:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.431403:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb size from disk: 8388608 -> 8388608 00002000:00010000:2.0:1769074780.431406:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.431409:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.431411:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.431414:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb blocks from disk: 8192 -> 8192 00000100:00100000:2.0:1769074780.431437:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+18:3651:x1855007625177088:12345-0@lo:2:osp-syn-1-0.0 Request processed in 88us (660us total) trans 17179874559 rc 0/0 00000100:00100000:2.0:1769074780.431444:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14278 00000100:00100000:2.0:1769074780.431455:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14279 00000100:00100000:2.0:1769074780.431462:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+17:3651:x1855007625176192:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.431520:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:2.0:1769074780.431523:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074772 -> 1769074772 00002000:00010000:2.0:1769074780.431525:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime from disk: 1769074770 -> 1769074770 00002000:00010000:2.0:1769074780.431527:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074772 -> 1769074772 00002000:00010000:2.0:1769074780.431529:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb blocks from disk: 2024 -> 2024 00000100:00100000:2.0:1769074780.431555:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+18:3651:x1855007625176192:12345-0@lo:2:osp-syn-1-0.0 Request processed in 94us (671us total) trans 17179874560 rc 0/0 00000100:00100000:2.0:1769074780.431561:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14279 00000100:00100000:2.0:1769074780.431572:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14280 00000100:00100000:2.0:1769074780.431578:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+17:3651:x1855007625176704:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.431624:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.431627:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.431630:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.431632:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.431634:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074780.431658:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+18:3651:x1855007625176704:12345-0@lo:2:osp-syn-1-0.0 Request processed in 81us (736us total) trans 17179874561 rc 0/0 00000100:00100000:2.0:1769074780.431664:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14280 00000004:00080000:2.0:1769074780.431719:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc700/2, rc 0, transno 17179874560 00000100:00100000:2.0:1769074780.431733:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625176192:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.431744:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dfb80/2, rc 0, transno 17179874561 00000100:00100000:2.0:1769074780.431747:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625176704:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.431757:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddc00/2, rc 0, transno 17179874558 00000100:00100000:2.0:1769074780.431759:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625176576:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.431766:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475df800/2, rc 0, transno 17179874559 00000100:00100000:2.0:1769074780.431769:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625177088:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.431862:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074780.431870:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549208e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625177216:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.431898:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.431922:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177216 00000100:00100000:2.0:1769074780.431927:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177216 00000100:00100000:2.0:1769074780.431953:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14281 00000100:00100000:2.0:1769074780.431959:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+18:3651:x1855007625177216:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.432015:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:2.0:1769074780.432018:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.432020:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.432023:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.432025:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:2.0:1769074780.432053:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+19:3651:x1855007625177216:12345-0@lo:2:osp-syn-1-0.0 Request processed in 94us (153us total) trans 17179874562 rc 0/0 00000100:00100000:2.0:1769074780.432059:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14281 00000004:00080000:2.0:1769074780.432081:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549208e00/2, rc 0, transno 17179874562 00000100:00100000:2.0:1769074780.432085:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549208e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625177216:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.432099:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176320:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.432124:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.432140:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176832:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.432156:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.432170:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1769074780.432175:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176448:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.432207:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.432220:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176960:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074780.432238:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074780.432260:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176320 00000100:00100000:2.0:1769074780.432263:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176320 00000100:00100000:2.0:1769074780.432282:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176832 00000100:00100000:2.0:1769074780.432286:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176832 00000100:00100000:2.0:1769074780.432301:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176448 00000100:00100000:2.0:1769074780.432303:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176448 00000100:00100000:2.0:1769074780.432320:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625176960 00000100:00100000:2.0:1769074780.432324:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625176960 00000100:00100000:2.0:1769074780.432334:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14282 00000100:00100000:2.0:1769074780.432338:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+22:3649:x1855007625176320:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.432394:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074780.432398:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.432401:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.432403:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074776 -> 1769074776 00002000:00010000:2.0:1769074780.432406:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:2.0:1769074780.432434:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+23:3649:x1855007625176320:12345-0@lo:2:osp-syn-1-0.0 Request processed in 95us (309us total) trans 17179874563 rc 0/0 00000100:00100000:2.0:1769074780.432440:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14282 00000100:00100000:2.0:1769074780.432454:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14283 00000100:00100000:2.0:1769074780.432460:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+22:3649:x1855007625176832:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.432509:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.432512:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.432514:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.432516:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.432517:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074780.432540:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+23:3649:x1855007625176832:12345-0@lo:2:osp-syn-1-0.0 Request processed in 80us (382us total) trans 17179874564 rc 0/0 00000100:00100000:2.0:1769074780.432546:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14283 00000100:00100000:2.0:1769074780.432558:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14284 00000100:00100000:2.0:1769074780.432564:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+22:3649:x1855007625176448:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.432607:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:2.0:1769074780.432610:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074777 -> 1769074777 00002000:00010000:2.0:1769074780.432613:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime from disk: 1769074777 -> 1769074777 00002000:00010000:2.0:1769074780.432615:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074777 -> 1769074777 00002000:00010000:2.0:1769074780.432617:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb blocks from disk: 5296 -> 5296 00000100:00100000:2.0:1769074780.432641:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+23:3649:x1855007625176448:12345-0@lo:2:osp-syn-1-0.0 Request processed in 77us (433us total) trans 17179874565 rc 0/0 00000100:00100000:2.0:1769074780.432647:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14284 00000100:00100000:2.0:1769074780.432660:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14285 00000100:00100000:2.0:1769074780.432666:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+22:3649:x1855007625176960:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074780.432710:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074780.432713:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.432715:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.432717:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.432719:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:2.0:1769074780.432744:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+23:3649:x1855007625176960:12345-0@lo:2:osp-syn-1-0.0 Request processed in 78us (505us total) trans 17179874566 rc 0/0 00000100:00100000:2.0:1769074780.432751:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14285 00000004:00080000:2.0:1769074780.432791:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddf80/2, rc 0, transno 17179874565 00000100:00100000:2.0:1769074780.432796:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176448:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.432804:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc000/2, rc 0, transno 17179874566 00000100:00100000:2.0:1769074780.432847:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176960:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.432857:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dca80/2, rc 0, transno 17179874563 00000100:00100000:2.0:1769074780.432861:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176320:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074780.432868:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475de300/2, rc 0, transno 17179874564 00000100:00100000:2.0:1769074780.432871:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625176832:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074780.434301:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625177728:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074780.434361:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074780.434389:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625178240:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074780.434400:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074780.434419:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1769074780.434424:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625177600:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074780.434441:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074780.434453:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625178112:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074780.434474:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.435040:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177472:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074780.435078:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.435111:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177984:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074780.435129:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.435145:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1769074780.435149:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177344:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074780.435161:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.435170:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177856:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074780.435196:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074780.435231:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177728 00000100:00100000:1.0:1769074780.435239:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177728 00000004:00080000:3.0:1769074780.435269:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1936 committed 00000004:00080000:3.0:1769074780.435273:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1936 committed 00000004:00080000:3.0:1769074780.435276:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1935 committed 00000004:00080000:3.0:1769074780.435281:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1935 committed 00000100:00100000:1.0:1769074780.435283:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625178240 00000004:00080000:3.0:1769074780.435283:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1934 committed 00000004:00080000:3.0:1769074780.435285:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1933 committed 00000100:00100000:1.0:1769074780.435286:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625178240 00000004:00080000:3.0:1769074780.435287:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1934 committed 00000004:00080000:3.0:1769074780.435289:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1933 committed 00000004:00080000:3.0:1769074780.435291:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1932 committed 00000004:00080000:3.0:1769074780.435294:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1931 committed 00000004:00080000:3.0:1769074780.435296:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1932 committed 00000100:00100000:1.0:1769074780.435299:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177600 00000100:00100000:1.0:1769074780.435301:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177600 00000100:00100000:1.0:1769074780.435309:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625178112 00000100:00100000:1.0:1769074780.435311:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625178112 00000001:00080000:3.0:1769074780.435311:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943998 is committed 00000100:00100000:1.0:1769074780.435318:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177472 00000100:00100000:1.0:1769074780.435320:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177472 00000001:00080000:3.0:1769074780.435323:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589943995 is committed 00000100:00100000:1.0:1769074780.435327:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177984 00000100:00100000:1.0:1769074780.435328:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177984 00000100:00100000:1.0:1769074780.435335:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177344 00000100:00100000:1.0:1769074780.435337:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177344 00000100:00100000:1.0:1769074780.435344:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625177856 00000100:00100000:1.0:1769074780.435346:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625177856 00000100:00100000:1.0:1769074780.435357:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14286 00000100:00100000:1.0:1769074780.435363:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+20:3652:x1855007625177728:12345-0@lo:2:osp-syn-0-0.0 00010000:00010000:3.0:1769074780.435394:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c171 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.435421:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.435424:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074780.435428:0:8855:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074780.435438:0:8855:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074780.435443:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074780.435450:0:8855:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074780.435455:0:8855:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074780.435460:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074780.435501:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd54361 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c171 expref: 224 pid: 8855 timeout: 0 lvb_type: 3 00002000:00010000:1.0:1769074780.435505:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074780.435508:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:1.0:1769074780.435510:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074780.435512:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:1.0:1769074780.435515:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00010000:00010000:3.0:1769074780.435519:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c171 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.435546:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fa00, rc 0) 00000100:00100000:3.0:1769074780.435605:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63001:x1855007616918528:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 24914us (24960us total) trans 8589944000 rc 0/0 00000100:00100000:3.0:1769074780.435613:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13353 00010000:00010000:3.0:1769074780.435648:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074780.435656:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000100:00100000:0.0:1769074780.435707:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14287 00000004:00200000:3.0:1769074780.435709:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:3.0:1769074780.435714:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000100:00100000:0.0:1769074780.435717:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+21:3652:x1855007625178240:12345-0@lo:2:osp-syn-0-0.0 00000004:00010000:3.0:1769074780.435737:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00002000:00010000:0.0:1769074780.435783:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00010000:00010000:3.0:1769074780.435783:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5437d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c17f expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.435798:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c17f expref: 8 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.435835:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074780.435838:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074780.435841:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88600, rc 0) 00000100:00100000:3.0:1769074780.435865:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+8:63006:x1855007616919168:12345-192.168.204.60@tcp:101:cat.0 Request processed in 15164us (15257us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.435872:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13355 00000100:00100000:3.0:1769074780.435897:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14288 00000100:00100000:3.0:1769074780.435905:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+22:3652:x1855007625177600:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.435906:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.435909:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.435911:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.435913:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074780.435954:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+23:3652:x1855007625178240:12345-0@lo:2:osp-syn-0-0.0 Request processed in 239us (1552us total) trans 21474841233 rc 0/0 00000100:00100000:0.0:1769074780.435961:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14287 00002000:00010000:3.0:1769074780.435966:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00000100:00100000:0.0:1769074780.435974:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14289 00000100:00100000:0.0:1769074780.435981:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+22:3652:x1855007625178112:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:3.0:1769074780.435987:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074777 -> 1769074777 00002000:00010000:3.0:1769074780.435989:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:3.0:1769074780.435991:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074777 -> 1769074777 00002000:00010000:3.0:1769074780.435993:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00002000:00010000:0.0:1769074780.436030:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074780.436033:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.436035:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.436037:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.436039:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074780.436064:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+23:3652:x1855007625178112:12345-0@lo:2:osp-syn-0-0.0 Request processed in 83us (1588us total) trans 21474841235 rc 0/0 00000100:00100000:0.0:1769074780.436070:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14289 00000100:00100000:0.0:1769074780.436082:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14290 00000100:00100000:0.0:1769074780.436088:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+22:3650:x1855007625177472:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.436135:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074780.436138:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.436140:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.436142:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074775 -> 1769074775 00002000:00010000:0.0:1769074780.436143:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074780.436172:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625177472:12345-0@lo:2:osp-syn-0-0.0 Request processed in 84us (1093us total) trans 21474841236 rc 0/0 00000100:00100000:0.0:1769074780.436189:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14290 00000100:00100000:0.0:1769074780.436200:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14291 00000100:00100000:0.0:1769074780.436206:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+22:3650:x1855007625177984:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.436252:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074780.436255:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.436257:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074780.436259:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:0.0:1769074780.436261:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074780.436291:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625177984:12345-0@lo:2:osp-syn-0-0.0 Request processed in 84us (1158us total) trans 21474841237 rc 0/0 00000100:00100000:0.0:1769074780.436297:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14291 00000100:00100000:0.0:1769074780.436308:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14292 00000100:00100000:0.0:1769074780.436314:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+22:3650:x1855007625177344:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.436339:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14293 00000100:00100000:2.0:1769074780.436348:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625177856:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074780.436358:0:8399:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074780.436360:0:8399:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.436362:0:8399:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.436364:0:8399:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074776 -> 1769074776 00002000:00010000:0.0:1769074780.436366:0:8399:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074780.436390:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+24:3650:x1855007625177344:12345-0@lo:2:osp-syn-0-0.0 Request processed in 75us (1226us total) trans 21474841238 rc 0/0 00000100:00100000:0.0:1769074780.436396:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14292 00002000:00010000:2.0:1769074780.436401:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:2.0:1769074780.436416:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.436418:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074780.436420:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:2.0:1769074780.436423:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000004:00080000:0.0:1769074780.436444:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460ec700/2, rc 0, transno 21474841235 00000100:00100000:0.0:1769074780.436450:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625178112:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074780.436451:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625177856:12345-0@lo:2:osp-syn-0-0.0 Request processed in 106us (1254us total) trans 21474841239 rc 0/0 00000100:00100000:2.0:1769074780.436456:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14293 00000004:00080000:0.0:1769074780.436462:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5462dce00/2, rc 0, transno 21474841233 00000100:00100000:0.0:1769074780.436465:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625178240:0@lo:2:osp-syn-0-0.0 00010000:00010000:0.0:1769074780.436480:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074780.436487:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074780.436517:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:0.0:1769074780.436520:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074780.436546:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074780.437242:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+22:3652:x1855007625177600:12345-0@lo:2:osp-syn-0-0.0 Request processed in 1336us (2797us total) trans 21474841234 rc 0/0 00000100:00100000:3.0:1769074780.437256:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14288 00000100:00100000:1.0:1769074780.437977:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+21:3652:x1855007625177728:12345-0@lo:2:osp-syn-0-0.0 Request processed in 2608us (3610us total) trans 21474841232 rc 0/0 00000100:00100000:1.0:1769074780.437986:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14286 00000004:00080000:1.0:1769074780.438197:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460efb80/2, rc 0, transno 21474841238 00000100:00100000:1.0:1769074780.438204:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177344:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074780.438216:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460ec000/2, rc 0, transno 21474841239 00000100:00100000:1.0:1769074780.438219:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177856:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074780.438229:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460edc00/2, rc 0, transno 21474841236 00000100:00100000:1.0:1769074780.438232:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177472:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074780.438241:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460ece00/2, rc 0, transno 21474841237 00000100:00100000:1.0:1769074780.438244:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625177984:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074780.439494:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460ef480/2, rc 0, transno 21474841234 00000100:00100000:2.0:1769074780.439506:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625177600:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074780.439518:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5460edf80/2, rc 0, transno 21474841232 00000100:00100000:2.0:1769074780.439521:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625177728:0@lo:2:osp-syn-0-0.0 00010000:00010000:0.0:1769074780.439919:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5436f lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 223 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.439945:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.439955:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.439958:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769074780.439961:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877e00, rc 0) 00000100:00100000:0.0:1769074780.440016:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63007:x1855007616918912:12345-192.168.204.60@tcp:101:cat.0 Request processed in 20425us (22288us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.440025:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13354 00000100:00100000:1.0:1769074780.442302:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.442479:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919424 00000100:00100000:0.0:1769074780.442489:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919424 00000100:00100000:0.0:1769074780.442555:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13356 00000100:00100000:0.0:1769074780.442691:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+8:63006:x1855007616919424:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.442712:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.442733:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5438b lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c186 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.442803:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5438b lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c186 expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.442841:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5438b lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c186 expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.442860:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074780.442864:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074780.442867:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876c00, rc 0) 00000100:00100000:0.0:1769074780.442932:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:63006:x1855007616919424:12345-192.168.204.60@tcp:101:cat.0 Request processed in 368us (639us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.442938:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13356 00000100:00100000:3.0:1769074780.443466:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.443543:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.443749:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.445035:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919680 00000100:00100000:1.0:1769074780.445046:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919680 00000100:00100000:1.0:1769074780.445175:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920320 00000100:00100000:1.0:1769074780.445190:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920320 00000100:00100000:1.0:1769074780.445216:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14294 00000100:00100000:1.0:1769074780.445224:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007616919680:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074780.445239:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.445255:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c194 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.445277:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.445279:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.445288:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.445302:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 13 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.445320:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e200, rc 0) 00000100:00100000:1.0:1769074780.445371:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2406:x1855007616919680:12345-192.168.204.60@tcp:101:cat.0 Request processed in 147us (1908us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.445378:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14294 00000100:00100000:1.0:1769074780.445397:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14295 00000100:00100000:1.0:1769074780.445421:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:63001:x1855007616920320:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:1.0:1769074780.445433:0:28954:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.445445:0:28954:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1b7 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.445464:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 15 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.445472:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.445474:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.445477:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2535424->2736127] (req 2535424->2736127) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 18 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.445510:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 15 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.445527:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 15 pid: 8399 timeout: 1868 lvb_type: 1 00000100:00100000:1.0:1769074780.445545:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625178368:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074780.445623:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643035780 going to sleep for 11 seconds 00000100:00080000:2.0:1769074780.446503:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074780.447349:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.452760:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919936 00000100:00100000:0.0:1769074780.452781:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919936 00000100:00100000:1.0:1769074780.453017:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14296 00000100:00100000:1.0:1769074780.453030:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616919936:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074780.453053:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.453072:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1a2 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.453099:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.453101:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.453112:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 20 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.453129:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 20 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.453143:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:1.0:1769074780.453232:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616919936:12345-192.168.204.60@tcp:101:cat.0 Request processed in 203us (9481us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.453239:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14296 00000100:00100000:0.0:1769074780.453825:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920192 00000100:00100000:0.0:1769074780.453832:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920192 00000100:00100000:1.0:1769074780.453938:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14297 00000100:00100000:1.0:1769074780.453947:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2406:x1855007616920192:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:1.0:1769074780.453958:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.453980:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1b0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.454003:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.454005:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.454012:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.454023:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 14 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.454033:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e000, rc 0) 00000100:00100000:1.0:1769074780.454063:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007616920192:12345-192.168.204.60@tcp:101:cat.0 Request processed in 117us (6723us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.454068:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14297 00000100:00100000:3.0:1769074780.454336:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.454919:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.454957:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.455003:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919552 00000100:00100000:0.0:1769074780.455011:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919552 00000100:00100000:0.0:1769074780.455047:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616919808 00000100:00100000:0.0:1769074780.455050:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616919808 00000100:00100000:0.0:1769074780.455069:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920064 00000100:00100000:0.0:1769074780.455071:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920064 00000100:00100000:0.0:1769074780.455089:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14298 00000100:00100000:0.0:1769074780.455096:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616919552:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.455109:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.455125:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c18d expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.455145:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.455147:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.455155:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.455168:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.455188:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:0.0:1769074780.455231:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616919552:12345-192.168.204.60@tcp:101:cat.0 Request processed in 133us (895us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.455238:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14298 00000100:00100000:0.0:1769074780.455253:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14299 00000100:00100000:0.0:1769074780.455259:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007616919808:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.455266:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.455275:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c19b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.455289:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.455291:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.455296:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 15 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.455306:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 15 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.455314:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:0.0:1769074780.455336:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616919808:12345-192.168.204.60@tcp:101:cat.0 Request processed in 77us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.455342:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14299 00000100:00100000:0.0:1769074780.455355:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14300 00000100:00100000:0.0:1769074780.455361:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616920064:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.455367:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.455375:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd543bc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1a9 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.455391:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.455401:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd543bc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a9 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.455415:0:27844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd543bc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a9 expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.455427:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876400, rc 0) 00000100:00100000:1.0:1769074780.455451:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:28954:1855007625178368:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074780.455454:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007616920064:12345-192.168.204.60@tcp:101:cat.0 Request processed in 94us (498us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074780.455459:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14300 00010000:00010000:1.0:1769074780.455477:0:28954:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2535424->2736127] (req 2535424->2736127) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.455487:0:28954:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f400, rc 0) 00000100:00100000:1.0:1769074780.455508:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:63001:x1855007616920320:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 10105us (11965us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.455514:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14295 00000100:00100000:3.0:1769074780.459019:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.459261:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920448 00000100:00100000:2.0:1769074780.459271:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920448 00000100:00100000:2.0:1769074780.459337:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14301 00000100:00100000:2.0:1769074780.459348:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2408:x1855007616920448:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074780.459365:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.459384:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd543c3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1be expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.459412:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.459415:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.459426:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd543c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1be expref: 6 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074780.459444:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd543c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1be expref: 6 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074780.459461:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866200, rc 0) 00000100:00100000:2.0:1769074780.459533:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+6:2408:x1855007616920448:12345-192.168.204.60@tcp:101:cat.0 Request processed in 187us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.459541:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14301 00000100:00100000:2.0:1769074780.460280:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.460314:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.460343:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.460369:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.460385:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.460411:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.460421:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920704 00000100:00100000:0.0:1769074780.460428:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920704 00000100:00100000:2.0:1769074780.460439:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921216 00000100:00100000:2.0:1769074780.460442:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921216 00000100:00100000:0.0:1769074780.460459:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920960 00010000:00010000:2.0:1769074780.460461:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 16 pid: 8399 timeout: 1868 lvb_type: 1 00000100:00100000:0.0:1769074780.460462:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920960 00000100:00100000:0.0:1769074780.460475:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920576 00000100:00100000:0.0:1769074780.460478:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920576 00000100:00100000:0.0:1769074780.460489:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616920832 00000100:00100000:2.0:1769074780.460489:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7143 00000100:00100000:0.0:1769074780.460491:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616920832 00000100:00100000:2.0:1769074780.460497:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+16:2409:x1855007616921216:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1769074780.460505:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921088 00000100:00100000:0.0:1769074780.460507:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921088 00010000:00010000:2.0:1769074780.460509:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1769074780.460524:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14302 00000100:00100000:0.0:1769074780.460530:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616920704:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.460541:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.460543:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 16 pid: 8399 timeout: 1868 lvb_type: 1 00010000:00010000:0.0:1769074780.460553:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd543ca lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1cc expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.460573:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.460575:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.460581:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd543ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1cc expref: 8 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.460594:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd543ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1cc expref: 8 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.460604:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877400, rc 0) 00000100:00100000:0.0:1769074780.460638:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616920704:12345-192.168.204.60@tcp:101:cat.0 Request processed in 108us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.460645:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14302 00000100:00100000:1.0:1769074780.460652:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14303 00000100:00100000:0.0:1769074780.460657:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14304 00000100:00100000:1.0:1769074780.460661:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616920960:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:0.0:1769074780.460662:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616920576:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.460668:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.460668:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.460675:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd543d1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1c5 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.460675:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fc00/0x7fe2323adfd543d8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1da expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.460686:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.460687:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.460688:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.460692:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd543d1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1c5 expref: 8 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.460696:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fc00/0x7fe2323adfd543d8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1da expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.460701:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd543d1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1c5 expref: 8 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.460710:0:28519:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0fc00/0x7fe2323adfd543d8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 6 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1da expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.460714:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877800, rc 0) 00010000:00010000:1.0:1769074780.460721:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0fc00, rc 0) 00000100:00100000:0.0:1769074780.460733:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616920576:12345-192.168.204.60@tcp:101:cat.0 Request processed in 71us (390us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.460735:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616920960:12345-192.168.204.60@tcp:101:cat.0 Request processed in 77us (422us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074780.460739:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14304 00000100:00100000:1.0:1769074780.460742:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14303 00000100:00100000:0.0:1769074780.460750:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14305 00000100:00100000:1.0:1769074780.460754:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14306 00000100:00100000:0.0:1769074780.460756:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+6:2408:x1855007616920832:12345-192.168.204.60@tcp:101:cat.0 00000100:00100000:1.0:1769074780.460759:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616921088:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:0.0:1769074780.460762:0:10284:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074780.460763:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074780.460771:0:10284:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd543e6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1d3 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.460781:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.460781:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd543df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1e1 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074780.460783:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074780.460788:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd543e6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1d3 expref: 7 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.460791:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.460792:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.460797:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd543df lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e1 expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.460799:0:10284:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd543e6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1d3 expref: 7 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074780.460831:0:10284:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00010000:00010000:1.0:1769074780.460834:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd543df lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e1 expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074780.460842:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0e800, rc 0) 00000100:00100000:0.0:1769074780.460850:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+7:2408:x1855007616920832:12345-192.168.204.60@tcp:101:cat.0 Request processed in 94us (480us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074780.460856:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14305 00000100:00100000:1.0:1769074780.460856:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616921088:12345-192.168.204.60@tcp:101:cat.0 Request processed in 97us (470us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.460865:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14306 00010000:00010000:2.0:1769074780.461117:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 16 pid: 8399 timeout: 1868 lvb_type: 1 00010000:00010000:2.0:1769074780.461131:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd541e0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0ec expref: 16 pid: 8399 timeout: 1868 lvb_type: 1 00010000:00010000:2.0:1769074780.461140:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.461141:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074780.461148:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.461156:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.461207:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074780.461224:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625178496:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074780.461276:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074780.463757:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.464112:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921344 00000100:00100000:3.0:1769074780.464125:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921344 00000100:00100000:3.0:1769074780.464224:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14307 00000100:00100000:3.0:1769074780.464234:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+7:63002:x1855007616921344:12345-192.168.204.60@tcp:101:multiop.0 00010000:00010000:3.0:1769074780.464253:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.464270:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1e8 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.464299:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.464307:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074780.464309:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074780.464314:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2367488->2854911] (req 2367488->2854911) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.464352:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 19 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.464372:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 19 pid: 28954 timeout: 1868 lvb_type: 0 00000100:00100000:3.0:1769074780.464386:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625178624:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074780.464555:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b033c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074780.474397:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625178496:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074780.474428:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074780.476275:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2409:x1855007616921216:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 15766us (15851us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.476291:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7143 00000100:00100000:1.0:1769074780.477212:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.477289:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921472 00000100:00100000:0.0:1769074780.477298:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921472 00002000:00100000:0.0:1769074780.477343:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2535424->2539519) req@ffff89b5436d5f80 x1855007616921472/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.477354:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d5f80 with x1855007616921472 ext(2535424->2539519) 00010000:00010000:0.0:1769074780.477363:0:28445:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.477377:0:28445:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 1868 lvb_type: 0 00002000:00010000:0.0:1769074780.477384:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5436d5f80 00000100:00100000:0.0:1769074780.477420:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 811 00000100:00100000:0.0:1769074780.477430:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616921472:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:0.0:1769074780.477605:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616921472:12345-192.168.204.60@tcp:3:cat.0 Request processed in 176us (397us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1769074780.477613:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 811 00002000:00100000:0.0:1769074780.477619:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2535424->2539519) req@ffff89b5436d5f80 x1855007616921472/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/4536 e 0 to 0 dl 1769074791 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.477626:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d5f80 with x1855007616921472 ext(2535424->2539519) 00010000:00010000:0.0:1769074780.477631:0:28445:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.477641:0:28445:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 1868 lvb_type: 0 00002000:00010000:0.0:1769074780.477647:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5436d5f80 00000100:00100000:3.0:1769074780.478957:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625178624:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074780.479166:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2367488->2854911] (req 2367488->2854911) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.479197:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:3.0:1769074780.479251:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+8:63002:x1855007616921344:12345-192.168.204.60@tcp:101:multiop.0 Request processed in 15019us (15510us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.479260:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14307 00000100:00100000:1.0:1769074780.496472:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.496533:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.496679:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.507720:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.511414:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921600 00000100:00100000:2.0:1769074780.511430:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921600 00002000:00100000:2.0:1769074780.512326:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5436d6a00 x1855007616921600/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.512339:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6a00 with x1855007616921600 ext(0->1048575) 00002000:00010000:2.0:1769074780.512346:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6a00 00000100:00100000:2.0:1769074780.512386:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921856 00000100:00100000:2.0:1769074780.512392:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921856 00002000:00100000:2.0:1769074780.512409:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54cf69f80 x1855007616921856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.512418:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf69f80 with x1855007616921856 ext(1048576->2097151) 00002000:00010000:2.0:1769074780.512422:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf69f80 00000100:00100000:2.0:1769074780.512443:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921728 00000100:00100000:2.0:1769074780.512446:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921728 00002000:00100000:2.0:1769074780.512460:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cf68700 x1855007616921728/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.512468:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68700 with x1855007616921728 ext(1048576->2097151) 00002000:00010000:2.0:1769074780.512471:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf68700 00000100:00100000:2.0:1769074780.512488:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8682 00000100:00100000:2.0:1769074780.512497:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616921600:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074780.513172:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8683 00000100:00100000:2.0:1769074780.513214:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616921856:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074780.513774:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8684 00000100:00100000:2.0:1769074780.513783:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616921728:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.525132:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.530930:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.533551:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616921984 00000100:00100000:3.0:1769074780.533572:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616921984 00002000:00100000:3.0:1769074780.533630:0:28446:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_014: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5499d0380 x1855007616921984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.533642:0:28446:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0380 with x1855007616921984 ext(1048576->2097151) 00002000:00010000:3.0:1769074780.533648:0:28446:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0380 00000100:00100000:3.0:1769074780.533678:0:28446:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616922112 00000100:00100000:3.0:1769074780.533683:0:28446:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616922112 00002000:00100000:3.0:1769074780.533697:0:28446:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_014: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2732032->2736127) req@ffff89b5499d1c00 x1855007616922112/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.533705:0:28446:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1c00 with x1855007616922112 ext(2732032->2736127) 00010000:00010000:3.0:1769074780.533712:0:28446:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.533728:0:28446:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00002000:00010000:3.0:1769074780.533734:0:28446:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d1c00 00000100:00100000:3.0:1769074780.533756:0:28446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 812 00000100:00100000:3.0:1769074780.533771:0:28446:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616922112:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074780.533970:0:28446:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007616922112:12345-192.168.204.60@tcp:3:cat.0 Request processed in 201us (8850us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1769074780.533978:0:28446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 812 00002000:00100000:3.0:1769074780.533985:0:28446:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_014: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2732032->2736127) req@ffff89b5499d1c00 x1855007616922112/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/4536 e 0 to 0 dl 1769074791 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.534009:0:28446:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d1c00 with x1855007616922112 ext(2732032->2736127) 00010000:00010000:3.0:1769074780.534015:0:28446:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.534025:0:28446:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00002000:00010000:3.0:1769074780.534036:0:28446:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5499d1c00 00000100:00100000:3.0:1769074780.534054:0:28446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8685 00000100:00100000:3.0:1769074780.534061:0:28446:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616921984:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.543538:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.543567:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.543587:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.543802:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616922240 00000100:00100000:0.0:1769074780.543848:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616922240 00002000:00100000:0.0:1769074780.543910:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b543858700 x1855007616922240/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.543922:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858700 with x1855007616922240 ext(0->1048575) 00002000:00010000:0.0:1769074780.543929:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543858700 00000100:00100000:0.0:1769074780.543957:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616922368 00000100:00100000:0.0:1769074780.543960:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616922368 00002000:00100000:0.0:1769074780.543973:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54385a300 x1855007616922368/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.543981:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a300 with x1855007616922368 ext(1048576->2097151) 00002000:00010000:0.0:1769074780.543985:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54385a300 00000100:00100000:0.0:1769074780.543995:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616922496 00000100:00100000:0.0:1769074780.543998:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616922496 00002000:00100000:0.0:1769074780.544012:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5460ee680 x1855007616922496/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.544020:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ee680 with x1855007616922496 ext(1048576->2097151) 00002000:00010000:0.0:1769074780.544023:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5460ee680 00000100:00100000:0.0:1769074780.544032:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616922624 00000100:00100000:0.0:1769074780.544035:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616922624 00002000:00100000:0.0:1769074780.544046:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5460ef800 x1855007616922624/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.544052:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef800 with x1855007616922624 ext(1048576->2097151) 00002000:00010000:0.0:1769074780.544055:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5460ef800 00000100:00100000:0.0:1769074780.544071:0:24031:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8686 00000100:00100000:0.0:1769074780.544081:0:24031:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616922240:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.545455:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8687 00000100:00100000:1.0:1769074780.545472:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2407:x1855007616922368:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074780.546433:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8688 00000100:00100000:3.0:1769074780.546444:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616922496:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074780.547140:0:28436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8689 00000100:00100000:3.0:1769074780.547150:0:28436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616922624:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074780.547867:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2407:x1855007616921856:12345-192.168.204.60@tcp:3:cat.0 Request processed in 34651us (51337us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074780.547879:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8683 00002000:00100000:3.0:1769074780.547892:0:28444:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b54cf69f80 x1855007616921856/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.547904:0:28444:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf69f80 with x1855007616921856 ext(1048576->2097151) 00002000:00010000:3.0:1769074780.547911:0:28444:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf69f80 00000100:00100000:3.0:1769074780.547998:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616921600:12345-192.168.204.60@tcp:3:cat.0 Request processed in 35502us (51540us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074780.548006:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8682 00002000:00100000:3.0:1769074780.548012:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b5436d6a00 x1855007616921600/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.548020:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5436d6a00 with x1855007616921600 ext(0->1048575) 00002000:00010000:3.0:1769074780.548023:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5436d6a00 00000100:00100000:2.0:1769074780.567666:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616921728:12345-192.168.204.60@tcp:3:cat.0 Request processed in 53871us (70974us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074780.567681:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8684 00002000:00100000:2.0:1769074780.567695:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b54cf68700 x1855007616921728/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.567705:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54cf68700 with x1855007616921728 ext(1048576->2097151) 00002000:00010000:2.0:1769074780.567711:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54cf68700 00010000:00010000:0.0:1769074780.573067:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074780.581552:0:28446:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_014:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616921984:12345-192.168.204.60@tcp:3:cat.0 Request processed in 47477us (73830us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074780.581566:0:28446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8685 00002000:00100000:3.0:1769074780.581578:0:28446:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_014: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b5499d0380 x1855007616921984/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.581589:0:28446:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5499d0380 with x1855007616921984 ext(1048576->2097151) 00002000:00010000:3.0:1769074780.581595:0:28446:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5499d0380 00000100:00100000:0.0:1769074780.582053:0:24031:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616922240:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37971us (51135us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074780.582061:0:24031:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8686 00002000:00100000:0.0:1769074780.582067:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x2bc:0x0] (0->1048575) req@ffff89b543858700 x1855007616922240/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.582075:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b543858700 with x1855007616922240 ext(0->1048575) 00002000:00010000:0.0:1769074780.582078:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b543858700 00000100:00100000:1.0:1769074780.596438:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.596706:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616922880 00000100:00100000:3.0:1769074780.596719:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616922880 00000100:00100000:3.0:1769074780.596785:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4692 00000100:00100000:3.0:1769074780.596797:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63001:x1855007616922880:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:3.0:1769074780.597135:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63001:x1855007616922880:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 340us (709us total) trans 8589944001 rc 0/0 00000100:00100000:3.0:1769074780.597142:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4692 00000100:00100000:3.0:1769074780.597219:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.597468:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923008 00000100:00100000:0.0:1769074780.597473:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923008 00002000:00100000:0.0:1769074780.597507:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2535424->2736127) req@ffff89b67ff0ea00 x1855007616923008/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.597528:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ea00 with x1855007616923008 ext(2535424->2736127) 00010000:00010000:0.0:1769074780.597538:0:24031:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:0.0:1769074780.597553:0:24031:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 20 pid: 28954 timeout: 1868 lvb_type: 0 00002000:00010000:0.0:1769074780.597561:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67ff0ea00 00000100:00100000:0.0:1769074780.597583:0:24031:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 813 00000100:00100000:0.0:1769074780.597590:0:24031:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616923008:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:0.0:1769074780.599661:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2407:x1855007616922368:12345-192.168.204.60@tcp:3:cat.0 Request processed in 54178us (56130us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074780.599684:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8687 00002000:00100000:0.0:1769074780.599697:0:28434:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x2c0000401:0x2be:0x0] (1048576->2097151) req@ffff89b54385a300 x1855007616922368/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.599707:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54385a300 with x1855007616922368 ext(1048576->2097151) 00002000:00010000:0.0:1769074780.599714:0:28434:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54385a300 00000100:00100000:1.0:1769074780.605377:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616922496:12345-192.168.204.60@tcp:3:cat.0 Request processed in 58923us (61796us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074780.605392:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8688 00002000:00100000:1.0:1769074780.605406:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x2ba:0x0] (1048576->2097151) req@ffff89b5460ee680 x1855007616922496/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.605417:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ee680 with x1855007616922496 ext(1048576->2097151) 00002000:00010000:1.0:1769074780.605424:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5460ee680 00000100:00100000:1.0:1769074780.606905:0:24031:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2406:x1855007616923008:12345-192.168.204.60@tcp:4:cat.0 Request processed in 9308us (9678us total) trans 17179874567 rc 0/0 00000100:00100000:1.0:1769074780.606922:0:24031:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 813 00002000:00100000:1.0:1769074780.606931:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2535424->2736127) req@ffff89b67ff0ea00 x1855007616923008/t17179874567(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/448 e 0 to 0 dl 1769074791 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.606942:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67ff0ea00 with x1855007616923008 ext(2535424->2736127) 00010000:00010000:1.0:1769074780.606951:0:24031:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 22 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.606968:0:24031:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 22 pid: 28954 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.606976:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b67ff0ea00 00000100:00100000:1.0:1769074780.610717:0:28436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2409:x1855007616922624:12345-192.168.204.60@tcp:3:cat.0 Request processed in 63551us (67112us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074780.610733:0:28436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8689 00002000:00100000:1.0:1769074780.610746:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_011: refresh rw locks for [0x280000401:0x2bb:0x0] (1048576->2097151) req@ffff89b5460ef800 x1855007616922624/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.610756:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5460ef800 with x1855007616922624 ext(1048576->2097151) 00002000:00010000:1.0:1769074780.610762:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5460ef800 00000100:00100000:0.0:1769074780.613586:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.613633:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.613673:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.613707:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.613730:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.613884:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923136 00000100:00100000:3.0:1769074780.613896:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923136 00000100:00100000:3.0:1769074780.613958:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14308 00000100:00100000:3.0:1769074780.613967:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:63007:x1855007616923136:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:3.0:1769074780.613987:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074780.614021:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1ef expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.614047:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074780.614054:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2367488->2854911] (req 2367488->2854911) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 7 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.614065:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 2 00010000:00010000:3.0:1769074780.614069:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.614115:0:8396:0:(ldlm_lockd.c:948:ldlm_server_blocking_ast()) ### lock not granted, not sending blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2367488->2854911] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 7 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.614128:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 23 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.614140:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:3.0:1769074780.614204:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:63007:x1855007616923136:12345-192.168.204.60@tcp:101:cat.0 Request processed in 237us (628us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.614211:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14308 00000100:00100000:3.0:1769074780.614235:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923520 00000100:00100000:3.0:1769074780.614239:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923520 00000100:00100000:3.0:1769074780.614271:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923392 00000100:00100000:3.0:1769074780.614274:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923392 00000100:00100000:3.0:1769074780.614289:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923648 00000100:00100000:3.0:1769074780.614291:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923648 00000100:00100000:3.0:1769074780.614305:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2329 00000100:00100000:3.0:1769074780.614311:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2409:x1855007616923520:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074780.614342:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2409:x1855007616923520:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 30us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.614348:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2329 00000100:00100000:3.0:1769074780.614359:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2330 00000100:00100000:3.0:1769074780.614364:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616923392:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074780.614385:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616923392:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 21us (678us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.614391:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2330 00000100:00100000:3.0:1769074780.614400:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2331 00000100:00100000:3.0:1769074780.614405:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2408:x1855007616923648:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074780.614426:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2408:x1855007616923648:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 21us (697us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.614432:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2331 00000100:00080000:3.0:1769074780.614449:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074780.615103:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923264 00000100:00100000:1.0:1769074780.615112:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923264 00000100:00100000:1.0:1769074780.615168:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 333 00000100:00100000:1.0:1769074780.615186:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2409:x1855007616923264:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074780.615235:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2409:x1855007616923264:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 59us (1601us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.615245:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 333 00000100:00100000:3.0:1769074780.615525:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.616801:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923776 00000100:00100000:1.0:1769074780.616851:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923776 00010000:00010000:1.0:1769074780.616894:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00000100:00100000:1.0:1769074780.616934:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7144 00000100:00100000:1.0:1769074780.616943:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616923776:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074780.616957:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769074780.616978:0:36300:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074777 -> 1769074780 00002000:00010000:1.0:1769074780.616981:0:36300:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074777 -> 1769074780 00010000:00010000:1.0:1769074780.616987:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.617000:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.617024:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f400/0x7fe2323adfd54399 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2535424->2736127) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b7 expref: 23 pid: 28954 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.617039:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.617042:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074780.617049:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 7 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.617059:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60080000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 7 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.617100:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 7 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074780.617113:0:36300:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x70000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 7 pid: 8396 timeout: 1868 lvb_type: 0 00000100:00100000:1.0:1769074780.617127:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625178880:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074780.617235:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442c480 going to sleep for 11 seconds 00000100:00100000:1.0:1769074780.627699:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625178880:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074780.627726:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074780.627789:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616923776:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 10845us (12269us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074780.627799:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7144 00000100:00100000:0.0:1769074780.632749:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.632931:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.633006:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924032 00000100:00100000:2.0:1769074780.633018:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924032 00000100:00100000:2.0:1769074780.633070:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14309 00000100:00100000:2.0:1769074780.633077:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+8:63006:x1855007616924032:12345-192.168.204.60@tcp:101:cat.0 00010000:00010000:2.0:1769074780.633094:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074780.633111:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1f6 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.633140:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074780.633145:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1f6 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.633153:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1f6 expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074780.633162:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b5495c8c00, rc 0) 00000100:00100000:2.0:1769074780.633234:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+9:63006:x1855007616924032:12345-192.168.204.60@tcp:101:cat.0 Request processed in 157us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074780.633241:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14309 00000100:00100000:1.0:1769074780.634167:0:28436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616923904 00000100:00100000:1.0:1769074780.634186:0:28436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616923904 00002000:00100000:1.0:1769074780.634227:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2367488->2371583) req@ffff89b5462dca80 x1855007616923904/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.634238:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dca80 with x1855007616923904 ext(2367488->2371583) 00010000:00010000:1.0:1769074780.634246:0:28436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.634260:0:28436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.634272:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5462dca80 00000100:00100000:1.0:1769074780.634350:0:28436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 814 00000100:00100000:1.0:1769074780.634361:0:28436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616923904:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.634530:0:28436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616923904:12345-192.168.204.60@tcp:3:cat.0 Request processed in 163us (1784us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074780.634537:0:28436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 814 00002000:00100000:1.0:1769074780.634544:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2367488->2371583) req@ffff89b5462dca80 x1855007616923904/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/4536 e 0 to 0 dl 1769074791 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.634553:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462dca80 with x1855007616923904 ext(2367488->2371583) 00010000:00010000:1.0:1769074780.634558:0:28436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.634567:0:28436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.634573:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5462dca80 00000100:00100000:0.0:1769074780.640223:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.640325:0:28436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924160 00000100:00100000:1.0:1769074780.640337:0:28436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924160 00002000:00100000:1.0:1769074780.640395:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2850816->2854911) req@ffff89b5462ddf80 x1855007616924160/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.640413:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462ddf80 with x1855007616924160 ext(2850816->2854911) 00010000:00010000:1.0:1769074780.640423:0:28436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.640440:0:28436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.640449:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5462ddf80 00000100:00100000:1.0:1769074780.640484:0:28436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 815 00000100:00100000:1.0:1769074780.640494:0:28436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616924160:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.640745:0:28436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2409:x1855007616924160:12345-192.168.204.60@tcp:3:cat.0 Request processed in 251us (530us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1769074780.640753:0:28436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 815 00002000:00100000:1.0:1769074780.640759:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2850816->2854911) req@ffff89b5462ddf80 x1855007616924160/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/4536 e 0 to 0 dl 1769074791 ref 1 fl Complete:H/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.640767:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462ddf80 with x1855007616924160 ext(2850816->2854911) 00010000:00010000:1.0:1769074780.640772:0:28436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.640782:0:28436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.640788:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5462ddf80 00000100:00100000:2.0:1769074780.645668:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.645787:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924544 00000100:00100000:0.0:1769074780.645797:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924544 00000100:00100000:0.0:1769074780.645882:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4693 00000100:00100000:0.0:1769074780.645891:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+9:63002:x1855007616924544:12345-192.168.204.60@tcp:35:multiop.0 00000100:00100000:0.0:1769074780.646160:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:63002:x1855007616924544:12345-192.168.204.60@tcp:35:multiop.0 Request processed in 268us (500us total) trans 8589944002 rc 0/0 00000100:00100000:0.0:1769074780.646168:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4693 00000100:00100000:0.0:1769074780.650620:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.650736:0:28436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924288 00000100:00100000:1.0:1769074780.650746:0:28436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924288 00002000:00100000:1.0:1769074780.650789:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2367488->2854911) req@ffff89b5462df480 x1855007616924288/t0(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.650798:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462df480 with x1855007616924288 ext(2367488->2854911) 00010000:00010000:1.0:1769074780.650834:0:28436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.650847:0:28436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 9 pid: 8396 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.650853:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5462df480 00000100:00100000:1.0:1769074780.650870:0:28436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 816 00000100:00100000:1.0:1769074780.650876:0:28436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+9:2408:x1855007616924288:12345-192.168.204.60@tcp:4:cat.0 00000100:00100000:1.0:1769074780.669645:0:28436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2408:x1855007616924288:12345-192.168.204.60@tcp:4:cat.0 Request processed in 18756us (19023us total) trans 17179874568 rc 0/0 00000100:00100000:1.0:1769074780.669661:0:28436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 816 00002000:00100000:1.0:1769074780.669675:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2367488->2854911) req@ffff89b5462df480 x1855007616924288/t17179874568(0) o4->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/448 e 0 to 0 dl 1769074791 ref 1 fl Complete:H/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.669685:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5462df480 with x1855007616924288 ext(2367488->2854911) 00010000:00010000:1.0:1769074780.669694:0:28436:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 11 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:1.0:1769074780.669714:0:28436:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 11 pid: 8396 timeout: 1868 lvb_type: 0 00002000:00010000:1.0:1769074780.669723:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b5462df480 00000100:00100000:1.0:1769074780.678633:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.678709:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924672 00000100:00100000:3.0:1769074780.678721:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924672 00010000:00010000:3.0:1769074780.678764:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 11 pid: 8396 timeout: 1868 lvb_type: 0 00000100:00100000:3.0:1769074780.678838:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7145 00000100:00100000:3.0:1769074780.678848:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2406:x1855007616924672:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074780.678864:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074780.678885:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x60000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 11 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.678899:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 11 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.678920:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd543ed lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2367488->2854911) gid 0 flags: 0x44801000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e8 expref: 11 pid: 8396 timeout: 1868 lvb_type: 0 00010000:00010000:3.0:1769074780.678936:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.678937:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.678944:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.678953:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.678960:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.678961:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074780.678965:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1f6 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.678985:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1f6 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074780.679031:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1f6 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074780.679045:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625179008:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074780.679136:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 21 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074780.679145:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625179136:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074780.679166:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e984780 going to sleep for 11 seconds 00000100:00100000:3.0:1769074780.682532:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625179008:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074780.684228:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:36300:1855007625179136:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074780.684259:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074780.684331:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2406:x1855007616924672:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5484us (5708us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074780.684342:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7145 00000100:00100000:2.0:1769074780.713753:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.713848:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.713886:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924800 00000100:00100000:0.0:1769074780.713899:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924800 00002000:00100000:0.0:1769074780.713960:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b549209880 x1855007616924800/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.713988:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209880 with x1855007616924800 ext(2097152->4194303) 00002000:00010000:0.0:1769074780.714000:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549209880 00000100:00100000:0.0:1769074780.714086:0:24031:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616924928 00000100:00100000:0.0:1769074780.714090:0:24031:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616924928 00002000:00100000:0.0:1769074780.714107:0:24031:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54920b480 x1855007616924928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074780.714115:0:24031:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920b480 with x1855007616924928 ext(2097152->4194303) 00002000:00010000:0.0:1769074780.714118:0:24031:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54920b480 00000100:00100000:3.0:1769074780.714925:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8690 00000100:00100000:3.0:1769074780.714941:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616924800:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.719956:0:28436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8691 00000100:00100000:2.0:1769074780.723276:0:28436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616924928:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:3.0:1769074780.723663:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.723703:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074780.726592:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616925056 00000100:00100000:1.0:1769074780.726609:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616925056 00002000:00100000:1.0:1769074780.726667:0:28434:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54510b100 x1855007616925056/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.726679:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510b100 with x1855007616925056 ext(2097152->4194303) 00002000:00010000:1.0:1769074780.726687:0:28434:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54510b100 00000100:00100000:1.0:1769074780.726720:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616925184 00000100:00100000:1.0:1769074780.726724:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616925184 00002000:00100000:1.0:1769074780.726741:0:28434:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b545109880 x1855007616925184/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.726749:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109880 with x1855007616925184 ext(2097152->4194303) 00002000:00010000:1.0:1769074780.726752:0:28434:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545109880 00000100:00100000:1.0:1769074780.726781:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8692 00000100:00100000:1.0:1769074780.726789:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616925056:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.728161:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8693 00000100:00100000:1.0:1769074780.740582:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616925184:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:1.0:1769074780.752429:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2408:x1855007616924800:12345-192.168.204.60@tcp:3:cat.0 Request processed in 37478us (38676us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074780.752445:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8690 00002000:00100000:1.0:1769074780.752463:0:16437:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b549209880 x1855007616924800/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.752474:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209880 with x1855007616924800 ext(2097152->4194303) 00002000:00010000:1.0:1769074780.752480:0:16437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549209880 00000100:00100000:1.0:1769074780.778154:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616925056:12345-192.168.204.60@tcp:3:cat.0 Request processed in 51347us (54476us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074780.778189:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8692 00002000:00100000:1.0:1769074780.778205:0:28434:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x2c0000401:0x2bf:0x0] (2097152->4194303) req@ffff89b54510b100 x1855007616925056/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074780.778215:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54510b100 with x1855007616925056 ext(2097152->4194303) 00002000:00010000:1.0:1769074780.778221:0:28434:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54510b100 00000100:00100000:3.0:1769074780.793653:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616925184:12345-192.168.204.60@tcp:3:cat.0 Request processed in 65468us (69935us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074780.793670:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8693 00002000:00100000:3.0:1769074780.793686:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b545109880 x1855007616925184/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.793697:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545109880 with x1855007616925184 ext(2097152->4194303) 00002000:00010000:3.0:1769074780.793707:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b545109880 00000100:00100000:2.0:1769074780.795916:0:28436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2408:x1855007616924928:12345-192.168.204.60@tcp:3:cat.0 Request processed in 75933us (82097us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074780.795929:0:28436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8691 00002000:00100000:2.0:1769074780.795941:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_011: refresh rw locks for [0x280000401:0x2bc:0x0] (2097152->4194303) req@ffff89b54920b480 x1855007616924928/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 2097152/2097152 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.795951:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54920b480 with x1855007616924928 ext(2097152->4194303) 00002000:00010000:2.0:1769074780.795956:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54920b480 00000100:00100000:1.0:1769074780.819508:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074780.819680:0:28436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616925312 00000100:00100000:2.0:1769074780.819692:0:28436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616925312 00002000:00100000:2.0:1769074780.819745:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5476fb100 x1855007616925312/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.819756:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fb100 with x1855007616925312 ext(4194304->4808703) 00002000:00010000:2.0:1769074780.819761:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476fb100 00000100:00100000:2.0:1769074780.819800:0:28436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8694 00000100:00100000:2.0:1769074780.819848:0:28436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616925312:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074780.827597:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.829004:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616925440 00000100:00100000:3.0:1769074780.829019:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616925440 00002000:00100000:3.0:1769074780.829069:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b549209f80 x1855007616925440/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/0 e 0 to 0 dl 1769074791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.829080:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209f80 with x1855007616925440 ext(4194304->4808703) 00002000:00010000:3.0:1769074780.829086:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549209f80 00000100:00100000:3.0:1769074780.829124:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8695 00000100:00100000:3.0:1769074780.829134:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616925440:12345-192.168.204.60@tcp:3:cat.0 00000100:00100000:2.0:1769074780.840380:0:28436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616925312:12345-192.168.204.60@tcp:3:cat.0 Request processed in 20522us (20870us total) trans 0 rc 614400/614400 00000100:00100000:2.0:1769074780.840395:0:28436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8694 00002000:00100000:2.0:1769074780.840407:0:28436:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b5476fb100 x1855007616925312/t0(0) o3->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074780.840418:0:28436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476fb100 with x1855007616925312 ext(4194304->4808703) 00002000:00010000:2.0:1769074780.840425:0:28436:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476fb100 00000100:00100000:3.0:1769074780.843547:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2409:x1855007616925440:12345-192.168.204.60@tcp:3:cat.0 Request processed in 14401us (16022us total) trans 0 rc 614400/614400 00000100:00100000:3.0:1769074780.843563:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8695 00002000:00100000:3.0:1769074780.843575:0:28445:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x2bf:0x0] (4194304->4808703) req@ffff89b549209f80 x1855007616925440/t0(0) o3->543044d2-7a16-4e58-9b2c-28bb782e5779@192.168.204.60@tcp:316/0 lens 488/440 e 0 to 0 dl 1769074791 ref 1 fl Complete:/600/0 rc 614400/614400 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769074780.843585:0:28445:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549209f80 with x1855007616925440 ext(4194304->4808703) 00002000:00010000:3.0:1769074780.843591:0:28445:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b549209f80 00000100:00100000:0.0:1769074780.862283:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074780.862331:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074780.862660:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616925696 00000100:00100000:3.0:1769074780.862672:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616925696 00000100:00100000:3.0:1769074780.862738:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616925952 00000100:00100000:3.0:1769074780.862742:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616925952 00000100:00100000:3.0:1769074780.862762:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4694 00000100:00100000:3.0:1769074780.862772:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63007:x1855007616925696:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074780.863101:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63007:x1855007616925696:12345-192.168.204.60@tcp:35:cat.0 Request processed in 330us (831us total) trans 8589944003 rc 0/0 00000100:00100000:3.0:1769074780.863110:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4694 00000100:00100000:3.0:1769074780.863127:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4695 00000100:00100000:3.0:1769074780.863134:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:63006:x1855007616925952:12345-192.168.204.60@tcp:35:cat.0 00000100:00100000:3.0:1769074780.863223:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+11:63006:x1855007616925952:12345-192.168.204.60@tcp:35:cat.0 Request processed in 90us (891us total) trans 8589944004 rc 0/0 00000100:00100000:3.0:1769074780.863229:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4695 00010000:00010000:2.0:1769074781.597177:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074782.623667:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074783.645073:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074783.884037:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074783.884083:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074783.884182:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074783.907855:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074783.907903:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616926848 00000100:00100000:0.0:1769074783.908027:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616926848 00000100:00100000:0.0:1769074783.908175:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14310 00000100:00100000:0.0:1769074783.908187:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2405:x1855007616926848:12345-192.168.204.60@tcp:9:umount.0 00000020:00000080:0.0:1769074783.908221:0:8396:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4ecb3 00000020:00010000:0.0:1769074783.908229:0:8396:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b5436f5800: total 0 00000100:00080000:0.0:1769074783.908234:0:8396:0:(import.c:65:import_set_state_nolock()) ffff89b5436f7800 : changing import state from FULL to CLOSED 00000001:00000004:0.0:1769074783.908286:0:8396:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 4, last_transno is 17179874568 00000100:00100000:0.0:1769074783.910761:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2405:x1855007616926848:12345-192.168.204.60@tcp:9:umount.0 Request processed in 2167us (2511us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074783.910783:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14310 00000100:00080000:0.0:1769074783.910847:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000020:00010000:0.0:1769074783.910861:0:8413:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b5436f5800: total 0 00010000:00010000:0.0:1769074783.910866:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074783.911156:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616926080 00000100:00100000:3.0:1769074783.911166:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616926080 00000100:00100000:3.0:1769074783.911223:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616926208 00000100:00100000:3.0:1769074783.911226:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616926208 00000100:00100000:3.0:1769074783.911242:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616926336 00000100:00100000:3.0:1769074783.911245:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616926336 00000100:00100000:3.0:1769074783.911263:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1028 00000100:00100000:3.0:1769074783.911271:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a7835f93-1e5d-43a4-8110-91f072d9ae65+17:2406:x1855007616926080:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074783.911285:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074783.911304:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b679924c00/0x7fe2323adfd4ec89 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 15 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798de6 expref: 17 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.911320:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074783.911707:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a7835f93-1e5d-43a4-8110-91f072d9ae65+16:2406:x1855007616926080:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 88us (27332us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074783.911717:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1028 00000100:00100000:3.0:1769074783.911736:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1029 00000100:00100000:3.0:1769074783.911744:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a7835f93-1e5d-43a4-8110-91f072d9ae65+15:2408:x1855007616926208:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074783.911752:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074783.911762:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b679924400/0x7fe2323adfd4ec90 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 7 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798e17 expref: 15 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.911772:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074783.911792:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a7835f93-1e5d-43a4-8110-91f072d9ae65+14:2408:x1855007616926208:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 50us (27709us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074783.911798:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1029 00000100:00100000:3.0:1769074783.911857:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1030 00000100:00100000:3.0:1769074783.911864:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a7835f93-1e5d-43a4-8110-91f072d9ae65+13:2406:x1855007616926336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074783.911873:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074783.911881:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b547876000/0x7fe2323adfd4ec97 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 9 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798e1e expref: 13 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.911890:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074783.911908:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2406:x1855007616926336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 45us (27726us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074783.912128:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1030 00010000:00010000:0.0:1769074783.912527:0:8413:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b5436f5800, canceled 0 locks, left on hash table 3. 00010000:00010000:0.0:1769074783.912584:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd543c3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1be expref: 7 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074783.912608:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8c00/0x7fe2323adfd543fb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1f6 expref: 6 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074783.912626:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44600/0x7fe2323adfd543e6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1d3 expref: 5 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074783.912636:0:8413:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b5436f5800, canceled 3 locks, left on hash table 0. 00010000:00010000:0.0:1769074783.912652:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074783.912654:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074783.912656:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074783.934295:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074783.934401:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616927360 00000100:00100000:3.0:1769074783.934411:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616927360 00000100:00100000:3.0:1769074783.934490:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14311 00000100:00100000:3.0:1769074783.934501:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+8:2405:x1855007616927360:12345-192.168.204.60@tcp:9:umount.0 00000020:00000080:3.0:1769074783.934530:0:28955:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4ecac 00000020:00010000:3.0:1769074783.934537:0:28955:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b5448d6800: total 0 00000100:00080000:3.0:1769074783.934542:0:28955:0:(import.c:65:import_set_state_nolock()) ffff89b67b10d000 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1769074783.934591:0:28955:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 5, last_transno is 21474841239 00000100:00100000:3.0:1769074783.934735:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:543044d2-7a16-4e58-9b2c-28bb782e5779+6:2405:x1855007616927360:12345-192.168.204.60@tcp:9:umount.0 Request processed in 236us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074783.934743:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14311 00000100:00080000:0.0:1769074783.934909:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000020:00010000:2.0:1769074783.935163:0:6502:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b5448d6800: total 0 00010000:00010000:2.0:1769074783.935199:0:6502:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b5448d6800, canceled 0 locks, left on hash table 3. 00010000:00010000:2.0:1769074783.935249:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877400/0x7fe2323adfd543ca lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1cc expref: 5 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074783.935274:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd543d1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1c5 expref: 4 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074783.935297:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e800/0x7fe2323adfd543df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1e1 expref: 3 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074783.935307:0:6502:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b5448d6800, canceled 3 locks, left on hash table 0. 00010000:00010000:2.0:1769074783.935318:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074783.935320:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074783.935322:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074783.946338:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074783.946460:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616927744 00000100:00100000:2.0:1769074783.946470:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616927744 00000100:00100000:2.0:1769074783.946551:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13357 00000100:00100000:2.0:1769074783.946562:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+11:2405:x1855007616927744:12345-192.168.204.60@tcp:39:umount.0 00000020:00000080:2.0:1769074783.946596:0:6514:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4ec9e 00000020:00010000:2.0:1769074783.946604:0:6514:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b549f55800: total 0 00000100:00080000:2.0:1769074783.946608:0:6514:0:(import.c:65:import_set_state_nolock()) ffff89b549f51800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1769074783.946639:0:6514:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 2, last_transno is 8589944004 00000100:00100000:2.0:1769074783.946875:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:543044d2-7a16-4e58-9b2c-28bb782e5779+10:2405:x1855007616927744:12345-192.168.204.60@tcp:39:umount.0 Request processed in 314us (548us total) trans 8589944005 rc 0/0 00000100:00100000:2.0:1769074783.946883:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13357 00000020:00010000:3.0:1769074783.947301:0:6501:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b549f55800: total 0 00010000:00010000:3.0:1769074783.947352:0:6501:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b549f55800, canceled 0 locks, left on hash table 4. 00010000:00010000:3.0:1769074783.947384:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074783.947387:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074783.947396:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd54384 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c17f expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.947417:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074783.947420:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074783.947425:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876c00/0x7fe2323adfd5438b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c186 expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.947436:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074783.947439:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1769074783.947442:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd4eed5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798f44 expref: 7 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.947455:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074783.947457:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074783.947461:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436400/0x7fe2323adfd4ed70 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798ee2 expref: 6 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074783.947470:0:6501:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b549f55800, canceled 4 locks, left on hash table 0. 00010000:00010000:3.0:1769074783.947481:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074783.947484:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074783.947485:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1769074783.947540:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074783.965403:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074783.976176:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616927872 00000100:00100000:1.0:1769074783.976196:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616927872 00000100:00100000:1.0:1769074783.976288:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13358 00000100:00100000:1.0:1769074783.976301:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+5:2405:x1855007616927872:12345-192.168.204.60@tcp:39:umount.0 00000020:00000080:1.0:1769074783.976336:0:6512:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4eca5 00000100:00080000:1.0:1769074783.976345:0:6512:0:(import.c:65:import_set_state_nolock()) ffff89b649256800 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1769074783.978139:0:6512:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-MDT0001_UUID: mount_count is 1, last_transno is 4294968345 00000100:00080000:0.0:1769074783.978477:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074783.978515:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:543044d2-7a16-4e58-9b2c-28bb782e5779+3:2405:x1855007616927872:12345-192.168.204.60@tcp:39:umount.0 Request processed in 2054us (12957us total) trans 4294968346 rc 0/0 00000100:00100000:1.0:1769074783.978528:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13358 00010000:00010000:3.0:1769074783.979067:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074784.157481:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625179520:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.157497:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625179264:0@lo:400:kworker.0 00000100:00100000:2.0:1769074784.157603:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625179392:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074784.162661:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074784.162751:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1769074784.162764:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625179648:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.162837:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074784.162860:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625180160:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074784.162886:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.167138:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.167234:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625179904:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.167292:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.167344:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625179904 00000100:00100000:1.0:1769074784.167353:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625179904 00000100:00100000:1.0:1769074784.167581:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1454 00000100:00100000:1.0:1769074784.167591:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625179904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.167650:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625179904:12345-0@lo:400:kworker.0 Request processed in 61us (358us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.167657:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1454 00000100:00080000:1.0:1769074784.167680:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074784.167717:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625179904:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.167762:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625179264 00000100:00100000:1.0:1769074784.167766:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625179264 00000100:00100000:1.0:1769074784.167794:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1769074784.167799:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625179264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.167855:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625179264:12345-0@lo:400:kworker.0 Request processed in 56us (738us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.167862:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1769074784.167898:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625179264:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.168886:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625179520 00000100:00100000:3.0:1769074784.168897:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625179520 00000100:00100000:3.0:1769074784.168946:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180160 00000100:00100000:3.0:1769074784.168950:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180160 00000100:00100000:3.0:1769074784.168970:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2651 00000100:00100000:3.0:1769074784.168978:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3651:x1855007625179520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.169094:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2652 00000100:00100000:1.0:1769074784.169104:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3651:x1855007625180160:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074784.169145:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445400576 avail 1299161088 00000100:00100000:2.0:1769074784.170704:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074784.170744:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625179776:0@lo:400:kworker.0 00000100:00100000:2.0:1769074784.170782:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074784.170853:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074784.170863:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625180032:0@lo:400:kworker.0 00000100:00100000:2.0:1769074784.170890:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074784.170932:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625179648 00000100:00100000:2.0:1769074784.170940:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625179648 00000100:00100000:2.0:1769074784.170985:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180032 00000100:00100000:2.0:1769074784.170989:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180032 00000100:00100000:2.0:1769074784.171018:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2332 00000100:00100000:2.0:1769074784.171026:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625179648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.185939:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3651:x1855007625179520:12345-0@lo:400:kworker.0 Request processed in 16948us (23273us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.185955:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2651 00000100:00100000:3.0:1769074784.185987:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625179392 00000100:00100000:3.0:1769074784.185997:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625179392 00000100:00100000:3.0:1769074784.186029:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2653 00000100:00100000:3.0:1769074784.186037:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625179392:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074784.186072:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074784.186149:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625179392:12345-0@lo:41:osp-pre-1-0.0 Request processed in 113us (15452us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.186155:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2653 00000100:00100000:3.0:1769074784.186195:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2333 00000100:00100000:3.0:1769074784.186205:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625180032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.186241:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625180032:12345-0@lo:400:kworker.0 Request processed in 39us (15349us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.186247:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2333 00000100:00100000:3.0:1769074784.186282:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625180544:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.186315:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.188536:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625180160:12345-0@lo:41:osp-pre-0-1.0 Request processed in 19423us (25637us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.188551:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2652 00000100:00080000:1.0:1769074784.188605:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074784.188624:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625179776 00000100:00100000:1.0:1769074784.188632:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625179776 00000100:00100000:1.0:1769074784.188691:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1455 00000100:00100000:1.0:1769074784.188700:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625179776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.188764:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625179776:12345-0@lo:400:kworker.0 Request processed in 65us (17980us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.188770:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1455 00000100:00080000:1.0:1769074784.188788:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074784.188799:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180544 00000100:00100000:1.0:1769074784.188802:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180544 00000100:00100000:1.0:1769074784.188848:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2334 00000100:00100000:1.0:1769074784.188855:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625180544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.188887:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625180544:12345-0@lo:400:kworker.0 Request processed in 33us (2572us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.188893:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2334 00000100:00080000:1.0:1769074784.189345:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074784.189451:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625180416:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.189495:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.189529:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074784.189535:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625180288:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.189555:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.189579:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180416 00000100:00100000:1.0:1769074784.189584:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180416 00000100:00100000:1.0:1769074784.189615:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180288 00000100:00100000:1.0:1769074784.189618:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180288 00000100:00100000:1.0:1769074784.189632:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2335 00000100:00100000:1.0:1769074784.189637:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625180416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.189667:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625180416:12345-0@lo:400:kworker.0 Request processed in 30us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.189672:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2335 00000100:00100000:1.0:1769074784.189685:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2336 00000100:00100000:1.0:1769074784.189690:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625180288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.189719:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625180288:12345-0@lo:400:kworker.0 Request processed in 29us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.189725:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2336 00000100:00080000:1.0:1769074784.189741:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074784.189743:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074784.189774:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625180288:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.189785:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625180416:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.189871:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625180032:0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.190235:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625179392:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074784.190254:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625179776:0@lo:400:kworker.0 00000100:00100000:2.0:1769074784.192130:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625179648:12345-0@lo:400:kworker.0 Request processed in 21082us (29270us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074784.192145:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2332 00000100:00100000:3.0:1769074784.193069:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625179648:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.193104:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625180160:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074784.193125:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625179520:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.193164:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625180544:0@lo:400:kworker.0 00000100:00100000:0.0:1769074784.196140:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074784.196163:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625180800:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.196197:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1769074784.196209:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625180672:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.196259:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074784.196300:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180672 00000100:00100000:3.0:1769074784.196308:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180672 00000100:00100000:3.0:1769074784.196362:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1456 00000100:00100000:3.0:1769074784.196369:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625180672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.196413:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625180672:12345-0@lo:400:kworker.0 Request processed in 44us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.196420:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1456 00000100:00080000:3.0:1769074784.196453:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074784.196500:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625180672:0@lo:400:kworker.0 00000100:00100000:0.0:1769074784.196543:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074784.197435:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625180928:0@lo:400:kworker.0 00000100:00100000:0.0:1769074784.197475:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074784.198053:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180928 00000100:00100000:1.0:1769074784.198063:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180928 00000100:00100000:1.0:1769074784.198126:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2337 00000100:00100000:1.0:1769074784.198135:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625180928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074784.198188:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625180928:12345-0@lo:400:kworker.0 Request processed in 54us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074784.198207:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2337 00000100:00100000:0.0:1769074784.199548:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625180928:0@lo:400:kworker.0 00000100:00100000:3.0:1769074784.204994:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625180800 00000100:00100000:3.0:1769074784.205009:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625180800 00000100:00100000:3.0:1769074784.205098:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1457 00000100:00100000:3.0:1769074784.205108:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625180800:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1769074784.205590:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074784.206127:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+23:3650:x1855007625180800:12345-0@lo:400:kworker.0 Request processed in 1008us (9571us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.206139:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1457 00000100:00100000:0.0:1769074784.210059:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625180800:0@lo:400:kworker.0 00000100:00100000:0.0:1769074784.669247:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625181056:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074784.669332:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074784.669382:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074784.669391:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625181184:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074784.669416:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074784.669500:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181056 00000100:00100000:3.0:1769074784.669508:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181056 00000100:00100000:3.0:1769074784.669559:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181184 00000100:00100000:3.0:1769074784.669563:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181184 00000100:00100000:3.0:1769074784.669597:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1172 00000100:00100000:3.0:1769074784.669607:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b84a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625181056:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074784.669642:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 638219584 pending 0 free 3905998848 avail 3674476544 00000100:00100000:3.0:1769074784.669694:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b84a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+21:3651:x1855007625181056:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (361us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.669702:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1172 00000100:00100000:3.0:1769074784.669959:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1173 00000100:00100000:3.0:1769074784.669966:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625181184:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074784.669976:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 638219584 pending 0 free 3905998848 avail 3674476544 00000100:00100000:3.0:1769074784.670001:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625181184:12345-0@lo:13:osp-pre-0-1.0 Request processed in 35us (583us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074784.670007:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1173 00000100:00100000:3.0:1769074784.670050:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625181184:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074784.670063:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625181056:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1769074784.671636:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074784.671701:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625181312:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074784.671768:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074784.671966:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181312 00000100:00100000:2.0:1769074784.671973:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181312 00000100:00100000:2.0:1769074784.672022:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1174 00000100:00100000:2.0:1769074784.672030:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+23:3649:x1855007625181312:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074784.672056:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 647443776 pending 737280 free 3900067840 avail 3668545536 00000100:00100000:2.0:1769074784.672123:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+23:3649:x1855007625181312:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074784.672130:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1174 00000100:00100000:3.0:1769074784.672534:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625181312:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074785.181225:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625181440:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074785.181306:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074785.181679:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181440 00000100:00100000:2.0:1769074785.181688:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181440 00000100:00100000:2.0:1769074785.181756:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1175 00000100:00100000:2.0:1769074785.181765:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625181440:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074785.181788:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 647443776 pending 737280 free 3900067840 avail 3668545536 00000100:00100000:2.0:1769074785.186069:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625181440:12345-0@lo:13:osp-pre-1-1.0 Request processed in 4293us (4750us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074785.186087:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1175 00000100:00100000:0.0:1769074785.186505:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625181440:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1769074785.693327:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074785.737549:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074785.737871:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074785.737904:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074785.737946:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074785.738107:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928384 00000100:00100000:3.0:1769074785.738119:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928384 00000100:00100000:3.0:1769074785.738181:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928896 00000100:00100000:3.0:1769074785.738184:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928896 00000100:00100000:3.0:1769074785.738208:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1458 00000100:00100000:3.0:1769074785.738217:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616928384:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074785.738307:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616928384:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 90us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074785.738315:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1458 00000100:00100000:3.0:1769074785.738334:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1459 00000100:00100000:3.0:1769074785.738341:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2407:x1855007616928896:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074785.738365:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2407:x1855007616928896:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (419us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074785.738372:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1459 00000100:00100000:3.0:1769074785.738442:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074785.738500:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074785.739367:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928128 00000100:00100000:0.0:1769074785.739376:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928128 00000100:00100000:0.0:1769074785.739428:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928640 00000100:00100000:0.0:1769074785.739432:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928640 00000100:00100000:0.0:1769074785.739450:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928256 00000100:00100000:0.0:1769074785.739453:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928256 00000100:00100000:0.0:1769074785.739470:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2338 00000100:00100000:0.0:1769074785.739477:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2407:x1855007616928128:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074785.739535:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2407:x1855007616928128:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 58us (1995us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074785.739541:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2338 00000100:00100000:0.0:1769074785.739555:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2339 00000100:00100000:0.0:1769074785.739561:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2407:x1855007616928640:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074785.739886:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074785.739931:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074785.739949:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074785.740104:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928768 00000100:00100000:2.0:1769074785.740110:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928768 00000100:00100000:2.0:1769074785.740139:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2340 00000100:00100000:2.0:1769074785.740146:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616928256:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074785.740186:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616928256:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 41us (1686us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074785.740192:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2340 00000100:00100000:2.0:1769074785.740205:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2341 00000100:00100000:2.0:1769074785.740211:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616928768:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074785.740245:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2407:x1855007616928768:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074785.740252:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2341 00000100:00100000:2.0:1769074785.740271:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928512 00000100:00100000:2.0:1769074785.740274:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928512 00000100:00100000:2.0:1769074785.740298:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616929024 00000100:00100000:2.0:1769074785.740301:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616929024 00000100:00100000:2.0:1769074785.740318:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1460 00000100:00100000:2.0:1769074785.740324:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616928512:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074785.740353:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616928512:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074785.740359:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1460 00000100:00100000:2.0:1769074785.740370:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1461 00000100:00100000:2.0:1769074785.740376:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2407:x1855007616929024:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074785.740401:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+15:2407:x1855007616929024:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074785.740407:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1461 00000100:00080000:3.0:1769074785.740465:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074785.740470:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074785.740473:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074785.740475:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074785.740621:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616928000 00000100:00100000:1.0:1769074785.740625:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616928000 00000100:00100000:1.0:1769074785.740653:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 335 00000100:00100000:1.0:1769074785.740660:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2407:x1855007616928000:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074785.740692:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2407:x1855007616928000:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (2250us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074785.740699:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 335 00000100:00100000:0.0:1769074785.745829:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2407:x1855007616928640:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 6240us (7898us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074785.745845:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2339 00000001:00100000:2.0:1769074786.207798:0:24714:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 543044d2-7a16-4e58-9b2c-28bb782e5779 00000020:00000080:2.0:1769074786.207832:0:24714:0:(genops.c:904:class_export_put()) final put ffff89b5448d6800/543044d2-7a16-4e58-9b2c-28bb782e5779 00000001:00080000:2.0:1769074786.207865:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841239 is committed 00000001:00080000:2.0:1769074786.207901:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841223 is committed 00000020:00000080:3.0:1769074786.207963:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b5448d6800/543044d2-7a16-4e58-9b2c-28bb782e5779 for lustre-OST0000 00000001:00100000:1.0:1769074786.208521:0:26036:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 543044d2-7a16-4e58-9b2c-28bb782e5779 00000001:00080000:1.0:1769074786.208532:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874568 is committed 00000020:00000080:1.0:1769074786.208538:0:26036:0:(genops.c:904:class_export_put()) final put ffff89b5436f5800/543044d2-7a16-4e58-9b2c-28bb782e5779 00000001:00080000:1.0:1769074786.208556:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874567 is committed 00000001:00080000:1.0:1769074786.208562:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874566 is committed 00000001:00080000:1.0:1769074786.208587:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874548 is committed 00000020:00000080:3.0:1769074786.212153:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b67b10d000 for lustre-OST0000 00000020:00000080:3.0:1769074786.212173:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b5436f5800/543044d2-7a16-4e58-9b2c-28bb782e5779 for lustre-OST0001 00000020:00000080:3.0:1769074786.212240:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b5436f7800 for lustre-OST0001 00000001:00100000:0.0:1769074786.212410:0:32377:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 543044d2-7a16-4e58-9b2c-28bb782e5779 00000001:00080000:0.0:1769074786.212422:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944005 is committed 00000001:00080000:0.0:1769074786.212431:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944003 is committed 00000020:00000080:0.0:1769074786.212435:0:32377:0:(genops.c:904:class_export_put()) final put ffff89b549f55800/543044d2-7a16-4e58-9b2c-28bb782e5779 00000020:00000080:1.0:1769074786.212519:0:35568:0:(genops.c:863:class_export_destroy()) destroying export ffff89b549f55800/543044d2-7a16-4e58-9b2c-28bb782e5779 for lustre-MDT0000 00000020:00000080:1.0:1769074786.212563:0:35568:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b549f51800 for lustre-MDT0000 00010000:00010000:0.0:1769074786.718127:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074787.157243:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.157299:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.157580:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616929152 00000100:00100000:2.0:1769074787.157593:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616929152 00000100:00100000:2.0:1769074787.157655:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616929280 00000100:00100000:2.0:1769074787.157659:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616929280 00000100:00100000:2.0:1769074787.157694:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1031 00000100:00100000:2.0:1769074787.157704:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a7835f93-1e5d-43a4-8110-91f072d9ae65+13:2409:x1855007616929152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074787.157720:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074787.157742:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b67b446e00/0x7fe2323adfd4ecba lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798e4f expref: 13 pid: 32381 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.157753:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1032 00010000:00010000:2.0:1769074787.157761:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074787.157768:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2409:x1855007616929280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074787.157779:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074787.157791:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b67b446a00/0x7fe2323adfd4ecc1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 6 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798e80 expref: 12 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.157803:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074787.157852:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a7835f93-1e5d-43a4-8110-91f072d9ae65+11:2409:x1855007616929152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 149us (617us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.157862:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1031 00000100:00100000:3.0:1769074787.157879:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:2409:x1855007616929280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 113us (576us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.157899:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1032 00000100:00100000:3.0:1769074787.161056:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.161280:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616929408 00000100:00100000:1.0:1769074787.161290:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616929408 00000100:00100000:1.0:1769074787.161354:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1033 00000100:00100000:1.0:1769074787.161364:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:2408:x1855007616929408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074787.161380:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074787.161400:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b547877a00/0x7fe2323adfd4ecc8 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 8 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798e87 expref: 10 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.161418:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074787.161480:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007616929408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 117us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.161488:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1033 00000100:00100000:0.0:1769074787.185942:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.186058:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616930560 00000100:00100000:1.0:1769074787.186080:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616930560 00000100:00100000:1.0:1769074787.186164:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14312 00000100:00100000:1.0:1769074787.186177:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2405:x1855007616930560:12345-192.168.204.60@tcp:9:umount.0 00000020:00000080:1.0:1769074787.186212:0:8398:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4ece4 00000020:00010000:1.0:1769074787.186219:0:8398:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b54361f000: total 0 00000100:00080000:1.0:1769074787.186223:0:8398:0:(import.c:65:import_set_state_nolock()) ffff89b54361d000 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1769074787.186275:0:8398:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 4, last_transno is 17179874568 00000100:00100000:1.0:1769074787.186458:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2405:x1855007616930560:12345-192.168.204.60@tcp:9:umount.0 Request processed in 284us (524us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.186475:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14312 00000020:00010000:0.0:1769074787.187682:0:6502:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b54361f000: total 0 00010000:00010000:0.0:1769074787.187690:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074787.187748:0:6502:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b54361f000, canceled 0 locks, left on hash table 8. 00010000:00010000:0.0:1769074787.187836:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867c00/0x7fe2323adfd542d5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c140 expref: 10 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.187875:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866600/0x7fe2323adfd541c4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0d0 expref: 9 pid: 8399 timeout: 0 lvb_type: 1 00000100:00080000:3.0:1769074787.188162:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074787.188219:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910cc00/0x7fe2323adfd50c17 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198799f89 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.188245:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876600/0x7fe2323adfd542dc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c147 expref: 7 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.188266:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866400/0x7fe2323adfd5429d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c116 expref: 6 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.188287:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866a00/0x7fe2323adfd542ce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c139 expref: 5 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.188754:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866c00/0x7fe2323adfd541d9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0e5 expref: 4 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074787.188770:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd542ab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c124 expref: 3 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.188781:0:6502:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b54361f000, canceled 8 locks, left on hash table 0. 00010000:00010000:0.0:1769074787.188927:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074787.188930:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074787.188932:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074787.223005:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.223667:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616931712 00000100:00100000:2.0:1769074787.223680:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616931712 00000100:00100000:2.0:1769074787.223853:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14313 00000100:00100000:2.0:1769074787.223865:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+13:2405:x1855007616931712:12345-192.168.204.60@tcp:9:umount.0 00000020:00000080:2.0:1769074787.223899:0:9048:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4ecdd 00000020:00010000:2.0:1769074787.223908:0:9048:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b5452d6800: total 0 00000100:00080000:2.0:1769074787.223913:0:9048:0:(import.c:65:import_set_state_nolock()) ffff89b543618800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1769074787.223970:0:9048:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 5, last_transno is 21474841239 00000100:00100000:2.0:1769074787.224178:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+11:2405:x1855007616931712:12345-192.168.204.60@tcp:9:umount.0 Request processed in 314us (1186us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.224188:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14313 00000020:00010000:0.0:1769074787.225472:0:6501:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b5452d6800: total 0 00010000:00010000:0.0:1769074787.225545:0:6501:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b5452d6800, canceled 0 locks, left on hash table 8. 00010000:00010000:0.0:1769074787.225609:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45200/0x7fe2323adfd54296 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c10f expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.225650:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ee00/0x7fe2323adfd541d2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0de expref: 9 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074787.225669:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45600/0x7fe2323adfd542a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c11d expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.225689:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877000/0x7fe2323adfd542b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c12b expref: 7 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.225711:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b546117000/0x7fe2323adfd5428f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c108 expref: 6 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.225738:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0ec00/0x7fe2323adfd541cb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0d7 expref: 5 pid: 10284 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074787.225755:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd50c48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198799fac expref: 4 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.225777:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b446800/0x7fe2323adfd541e7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c0f3 expref: 3 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074787.225788:0:6501:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b5452d6800, canceled 8 locks, left on hash table 0. 00010000:00010000:0.0:1769074787.225800:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074787.225802:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074787.225856:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074787.229513:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625181696:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229596:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229661:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625182208:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229685:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229703:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625182720:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229724:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229750:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:2.0:1769074787.229756:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625181568:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229785:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229832:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625182080:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229860:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229880:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625182592:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229900:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229915:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625183104:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.229936:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.229967:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181696 00000100:00100000:2.0:1769074787.229974:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181696 00000100:00100000:2.0:1769074787.230016:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182720 00000100:00100000:2.0:1769074787.230020:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182720 00000100:00100000:2.0:1769074787.230042:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2654 00000100:00100000:2.0:1769074787.230051:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625181696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230123:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625181696:12345-0@lo:400:kworker.0 Request processed in 72us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.230131:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2654 00000100:00100000:2.0:1769074787.230149:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2655 00000100:00100000:2.0:1769074787.230155:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625182720:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230183:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625182720:12345-0@lo:400:kworker.0 Request processed in 29us (459us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.230189:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2655 00000100:00100000:2.0:1769074787.230236:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625181696:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230247:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625182720:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230293:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625181824:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230332:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.230349:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625182336:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230366:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.230382:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625182848:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230400:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.230418:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074787.230423:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625181952:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230443:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.230460:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625182464:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230481:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.230490:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625182976:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.230509:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.230542:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182208 00000100:00100000:2.0:1769074787.230547:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182208 00000100:00100000:0.0:1769074787.230638:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182592 00000100:00100000:0.0:1769074787.230643:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182592 00000100:00100000:1.0:1769074787.231856:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181568 00000100:00100000:1.0:1769074787.231865:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181568 00000100:00100000:1.0:1769074787.231915:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:1.0:1769074787.231923:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b08c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625181568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074787.231995:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b08c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625181568:12345-0@lo:400:kworker.0 Request processed in 73us (2208us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.232003:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:1.0:1769074787.232037:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625183104 00000100:00100000:1.0:1769074787.232040:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625183104 00000100:00100000:1.0:1769074787.232088:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181824 00000100:00100000:1.0:1769074787.232092:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181824 00000100:00100000:1.0:1769074787.232104:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182336 00000100:00100000:1.0:1769074787.232107:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182336 00000100:00080000:3.0:1769074787.232266:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074787.232289:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074787.232292:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074787.232330:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182080 00000100:00100000:3.0:1769074787.232335:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182080 00000100:00100000:3.0:1769074787.232367:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182848 00000100:00100000:3.0:1769074787.232370:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182848 00000100:00100000:3.0:1769074787.232388:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625181952 00000100:00100000:3.0:1769074787.232390:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625181952 00000100:00100000:3.0:1769074787.232408:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182976 00000100:00100000:3.0:1769074787.232410:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182976 00000100:00100000:3.0:1769074787.232429:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1462 00000100:00100000:3.0:1769074787.232436:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625182080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.232477:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625182080:12345-0@lo:400:kworker.0 Request processed in 42us (2616us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.232483:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1462 00000100:00100000:3.0:1769074787.232495:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1463 00000100:00100000:3.0:1769074787.232501:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625182848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.232534:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625182848:12345-0@lo:400:kworker.0 Request processed in 33us (2133us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.232555:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1463 00000100:00100000:3.0:1769074787.232568:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1464 00000100:00100000:3.0:1769074787.232574:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625181952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.232600:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625181952:12345-0@lo:400:kworker.0 Request processed in 27us (2157us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.232607:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1464 00000100:00100000:3.0:1769074787.232615:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1465 00000100:00100000:3.0:1769074787.232621:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625182976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.232647:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625182976:12345-0@lo:400:kworker.0 Request processed in 26us (2137us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.232653:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1465 00000100:00080000:3.0:1769074787.232669:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074787.232672:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074787.232721:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625181952:0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.232732:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874566 gen 2 00000004:00080000:3.0:1769074787.232737:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf69880, transno 17179874549 00000004:00080000:3.0:1769074787.232746:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460eed80, transno 17179874550 00000004:00080000:3.0:1769074787.232750:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf6b100, transno 17179874551 00000004:00080000:3.0:1769074787.232753:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf68380, transno 17179874552 00000004:00080000:3.0:1769074787.232755:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf68000, transno 17179874553 00000004:00080000:3.0:1769074787.232758:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf68a80, transno 17179874554 00000004:00080000:3.0:1769074787.232761:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf69c00, transno 17179874555 00000004:00080000:3.0:1769074787.232763:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54510bb80, transno 17179874556 00000004:00080000:3.0:1769074787.232766:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54cf69180, transno 17179874557 00000004:00080000:3.0:1769074787.232768:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddc00, transno 17179874558 00000004:00080000:3.0:1769074787.232771:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df800, transno 17179874559 00000004:00080000:3.0:1769074787.232773:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc700, transno 17179874560 00000004:00080000:3.0:1769074787.232776:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dfb80, transno 17179874561 00000004:00080000:3.0:1769074787.232779:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549208e00, transno 17179874562 00000004:00080000:3.0:1769074787.232781:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dca80, transno 17179874563 00000004:00080000:3.0:1769074787.232784:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475de300, transno 17179874564 00000004:00080000:3.0:1769074787.232786:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddf80, transno 17179874565 00000004:00080000:3.0:1769074787.232789:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc000, transno 17179874566 00000100:00100000:3.0:1769074787.232794:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625182976:0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.232803:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841239 gen 2 00000004:00080000:3.0:1769074787.232832:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b543858e00, transno 21474841224 00000004:00080000:3.0:1769074787.232839:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5476fa680, transno 21474841225 00000004:00080000:3.0:1769074787.232842:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54385aa00, transno 21474841226 00000004:00080000:3.0:1769074787.232844:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5476f8380, transno 21474841227 00000004:00080000:3.0:1769074787.232847:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54385ad80, transno 21474841228 00000004:00080000:3.0:1769074787.232849:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5476f8e00, transno 21474841229 00000004:00080000:3.0:1769074787.232852:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b543859880, transno 21474841230 00000004:00080000:3.0:1769074787.232855:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5476fb480, transno 21474841231 00000004:00080000:3.0:1769074787.232858:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460edf80, transno 21474841232 00000004:00080000:3.0:1769074787.232861:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5462dce00, transno 21474841233 00000004:00080000:3.0:1769074787.232863:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ef480, transno 21474841234 00000004:00080000:3.0:1769074787.232866:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ec700, transno 21474841235 00000004:00080000:3.0:1769074787.232868:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460edc00, transno 21474841236 00000004:00080000:3.0:1769074787.232871:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ece00, transno 21474841237 00000004:00080000:3.0:1769074787.232873:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460efb80, transno 21474841238 00000004:00080000:3.0:1769074787.232875:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5460ec000, transno 21474841239 00000100:00100000:3.0:1769074787.232880:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625182848:0@lo:400:kworker.0 00000004:00080000:3.0:1769074787.233157:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 18 done 00000100:00100000:3.0:1769074787.233212:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625182464 00000100:00100000:3.0:1769074787.233218:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625182464 00000100:00100000:3.0:1769074787.233251:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2342 00000100:00100000:3.0:1769074787.233256:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625183104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233298:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625183104:12345-0@lo:400:kworker.0 Request processed in 42us (3361us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.233304:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2342 00000100:00100000:3.0:1769074787.233317:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2343 00000100:00100000:3.0:1769074787.233323:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625181824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233358:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625181824:12345-0@lo:400:kworker.0 Request processed in 34us (3024us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.233363:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2343 00000100:00100000:3.0:1769074787.233375:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2344 00000100:00100000:3.0:1769074787.233381:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625182464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233408:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625182464:12345-0@lo:400:kworker.0 Request processed in 28us (2926us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.233413:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2344 00000100:00100000:3.0:1769074787.233466:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625182464:0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233477:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625181824:0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233507:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625181568:0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233519:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625182080:0@lo:400:kworker.0 00000100:00100000:3.0:1769074787.233528:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625183104:0@lo:400:kworker.0 00000100:00080000:3.0:1769074787.233548:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074787.233632:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.233653:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932096 00000100:00100000:3.0:1769074787.233658:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932096 00000100:00100000:3.0:1769074787.233692:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13359 00000100:00100000:3.0:1769074787.233698:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+7:2405:x1855007616932096:12345-192.168.204.60@tcp:39:umount.0 00000020:00000080:3.0:1769074787.233721:0:6514:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4eccf 00000020:00010000:3.0:1769074787.233728:0:6514:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b67b10c000: total 0 00000100:00080000:3.0:1769074787.233733:0:6514:0:(import.c:65:import_set_state_nolock()) ffff89b67b10c800 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1769074787.233759:0:6514:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 2, last_transno is 8589944005 00000100:00100000:3.0:1769074787.233895:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+6:2405:x1855007616932096:12345-192.168.204.60@tcp:39:umount.0 Request processed in 196us (265us total) trans 8589944006 rc 0/0 00000100:00100000:3.0:1769074787.233903:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13359 00000100:00080000:3.0:1769074787.234010:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000020:00010000:3.0:1769074787.234027:0:8413:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b67b10c000: total 0 00010000:00010000:3.0:1769074787.234083:0:8413:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b67b10c000, canceled 0 locks, left on hash table 2. 00010000:00010000:3.0:1769074787.234119:0:8413:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074787.234123:0:8413:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1769074787.234132:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876800/0x7fe2323adfd50b92 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198799f58 expref: 5 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.234153:0:8413:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074787.234156:0:8413:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074787.234160:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877600/0x7fe2323adfd4ecf2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798eb8 expref: 4 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.234172:0:8413:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b67b10c000, canceled 2 locks, left on hash table 0. 00010000:00010000:3.0:1769074787.234183:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074787.234186:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074787.234188:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1769074787.234308:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 16 done 00000100:00100000:2.0:1769074787.235123:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2345 00000100:00100000:2.0:1769074787.235135:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625182208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.235260:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625182208:12345-0@lo:400:kworker.0 Request processed in 128us (5572us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.235268:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2345 00000100:00100000:2.0:1769074787.235488:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625182208:0@lo:400:kworker.0 00000100:00100000:0.0:1769074787.235731:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2346 00000100:00100000:0.0:1769074787.235740:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625182592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074787.235784:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625182592:12345-0@lo:400:kworker.0 Request processed in 46us (5883us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.235789:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2346 00000100:00100000:1.0:1769074787.235918:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2347 00000100:00100000:1.0:1769074787.235925:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625182336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074787.235965:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625182336:12345-0@lo:400:kworker.0 Request processed in 41us (5598us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.235970:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2347 00000100:00100000:1.0:1769074787.235998:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625182336:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.236137:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625182592:0@lo:400:kworker.0 00000100:00100000:2.0:1769074787.249871:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.250153:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932224 00000100:00100000:3.0:1769074787.250165:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932224 00000100:00100000:3.0:1769074787.250262:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13360 00000100:00100000:3.0:1769074787.250273:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+5:2405:x1855007616932224:12345-192.168.204.60@tcp:39:umount.0 00000100:00080000:0.0:1769074787.250685:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000020:00000080:3.0:1769074787.250738:0:6512:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd4ecd6 00000100:00080000:3.0:1769074787.250747:0:6512:0:(import.c:65:import_set_state_nolock()) ffff89b5452d0800 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1769074787.250832:0:6512:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-MDT0001_UUID: mount_count is 1, last_transno is 4294968346 00000100:00100000:3.0:1769074787.251003:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b+3:2405:x1855007616932224:12345-192.168.204.60@tcp:39:umount.0 Request processed in 730us (1141us total) trans 4294968347 rc 0/0 00000100:00100000:3.0:1769074787.251012:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13360 00010000:00010000:0.0:1769074787.256890:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074787.374949:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.376170:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932352 00000100:00100000:1.0:1769074787.376185:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932352 00000100:00100000:2.0:1769074787.376689:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13361 00000100:00100000:2.0:1769074787.376700:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63053:x1855007616932352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.376721:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074787.376742:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54402 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c1fd expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.376830:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.376845:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.376856:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.376871:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00000100:00080000:0.0:1769074787.376993:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1769074787.377086:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377098:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377199:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377210:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377217:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074787.377266:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:2.0:1769074787.377271:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074787.377291:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377640:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377651:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377658:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377665:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377673:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377680:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074787.377684:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377692:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377705:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074787.377708:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074787.377713:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377719:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54409 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377734:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377741:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377747:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377752:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074787.377755:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377761:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377768:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b64effad00) 00010000:00010000:2.0:1769074787.377771:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074787.377774:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377780:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54410 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.377932:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54402 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.377948:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.377955:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.377957:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074787.377960:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:2.0:1769074787.378043:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616932352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1344us (3109us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.378052:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13361 00000100:00100000:2.0:1769074787.389120:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074787.393281:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932608 00000100:00100000:0.0:1769074787.393298:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932608 00000100:00100000:0.0:1769074787.393835:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13362 00000100:00100000:0.0:1769074787.393848:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616932608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074787.393872:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074787.393903:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd5441e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c204 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.394035:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.394052:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 224 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.394071:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.394081:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 5/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.394117:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.394131:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 224 pid: 8855 timeout: 1875 lvb_type: 0 00000100:00100000:0.0:1769074787.394154:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625183232:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074787.397239:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 224 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.397262:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 224 pid: 6514 timeout: 1875 lvb_type: 0 00000100:00100000:0.0:1769074787.397285:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625183360:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074787.397360:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074787.406871:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074787.407115:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.407950:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932736 00000100:00100000:3.0:1769074787.407962:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932736 00010000:00010000:3.0:1769074787.408010:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 225 pid: 8855 timeout: 1875 lvb_type: 0 00000100:00100000:3.0:1769074787.408289:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932864 00000100:00100000:3.0:1769074787.408300:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932864 00010000:00010000:3.0:1769074787.408318:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 226 pid: 6514 timeout: 1875 lvb_type: 0 00000100:00100000:0.0:1769074787.408415:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7146 00000100:00100000:0.0:1769074787.408427:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2408:x1855007616932736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074787.408444:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074787.408451:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 226 pid: 8855 timeout: 1875 lvb_type: 0 00010000:00010000:0.0:1769074787.408465:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 226 pid: 8855 timeout: 1875 lvb_type: 0 00010000:00010000:0.0:1769074787.408488:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074787.408493:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.408503:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.408509:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074787.408581:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:2408:x1855007616932736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 156us (1720us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.408590:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7146 00000100:00100000:0.0:1769074787.408611:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7147 00000100:00100000:0.0:1769074787.408618:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:2408:x1855007616932864:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074787.408630:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074787.408635:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 225 pid: 6514 timeout: 1875 lvb_type: 0 00010000:00010000:0.0:1769074787.408645:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 225 pid: 6514 timeout: 1875 lvb_type: 0 00010000:00010000:0.0:1769074787.408656:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074787.408661:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.408669:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.408676:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074787.408692:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074787.408710:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:2408:x1855007616932864:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 93us (1596us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.408717:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7147 00010000:00010000:2.0:1769074787.411009:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54417 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1fd expref: 224 pid: 8855 timeout: 1875 lvb_type: 0 00000100:00100000:2.0:1769074787.411037:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625183232:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074787.411052:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54376 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c178 expref: 223 pid: 6514 timeout: 1875 lvb_type: 0 00000100:00100000:2.0:1769074787.411075:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625183360:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074787.411099:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.411110:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.411120:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074787.411151:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:2.0:1769074787.411682:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71a00/0x7fe2323adfd5441e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.411702:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.411713:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.411716:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1769074787.411720:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f70c00, rc 0) 00000100:00100000:2.0:1769074787.411796:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616932608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 17950us (22686us total) trans 8589944007 rc 0/0 00000100:00100000:2.0:1769074787.411957:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13362 00000100:00100000:0.0:1769074787.416791:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.416986:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616932992 00000100:00100000:2.0:1769074787.416997:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616932992 00000100:00100000:2.0:1769074787.417103:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13363 00000100:00100000:2.0:1769074787.417113:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616932992:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.417136:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074787.417170:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54433 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c20b expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.417262:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.417278:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.417287:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 4/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.417331:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.417343:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 224 pid: 8855 timeout: 1875 lvb_type: 0 00000100:00100000:2.0:1769074787.417355:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625183488:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074787.417445:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074787.421000:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625183488:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074787.421029:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000100:00100000:0.0:1769074787.432609:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.432767:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616933120 00000100:00100000:1.0:1769074787.432779:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616933120 00010000:00010000:1.0:1769074787.432855:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 225 pid: 8855 timeout: 1875 lvb_type: 0 00000100:00100000:1.0:1769074787.432900:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7148 00000100:00100000:1.0:1769074787.432909:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:2409:x1855007616933120:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074787.432924:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074787.432931:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 225 pid: 8855 timeout: 1875 lvb_type: 0 00010000:00010000:1.0:1769074787.432946:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 225 pid: 8855 timeout: 1875 lvb_type: 0 00010000:00010000:1.0:1769074787.432969:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54425 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x1fb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c204 expref: 225 pid: 8855 timeout: 1875 lvb_type: 0 00010000:00010000:1.0:1769074787.432986:0:33102:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074787.432991:0:33102:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x13/0x8 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074787.433002:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074787.433010:0:33102:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074787.433047:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074787.433128:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:2409:x1855007616933120:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 220us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.433139:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7148 00010000:00010000:2.0:1769074787.436408:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.436427:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074787.436510:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:2.0:1769074787.436516:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074787.436554:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.436612:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54433 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c20b expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.436638:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c20b expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.436646:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.436649:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074787.436653:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba400, rc 0) 00000100:00100000:2.0:1769074787.436738:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616932992:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 19625us (19956us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.436747:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13363 00000100:00100000:0.0:1769074787.457363:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.457915:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616933376 00000100:00100000:2.0:1769074787.457929:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616933376 00000100:00100000:2.0:1769074787.457998:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13364 00000100:00100000:2.0:1769074787.458009:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616933376:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.458029:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074787.458055:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54441 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c212 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.458174:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54448 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.458191:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54448 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.458203:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54448 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074787.458216:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:2.0:1769074787.458326:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54441 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c212 expref: 226 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.458346:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54448 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c212 expref: 225 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.458355:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.458358:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074787.458362:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:2.0:1769074787.458865:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63053:x1855007616933376:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 800us (1454us total) trans 8589944008 rc 0/0 00000100:00100000:2.0:1769074787.458873:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13364 00000100:00100000:1.0:1769074787.466911:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.466956:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.466995:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.467033:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.467055:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.467602:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616933504 00000100:00100000:2.0:1769074787.467616:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616933504 00000100:00100000:2.0:1769074787.467693:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616933632 00000100:00100000:2.0:1769074787.467698:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616933632 00000100:00100000:2.0:1769074787.467714:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616933760 00000100:00100000:2.0:1769074787.467717:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616933760 00000100:00100000:2.0:1769074787.467730:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616933888 00000100:00100000:2.0:1769074787.467733:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616933888 00000100:00100000:2.0:1769074787.467746:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934016 00000100:00100000:2.0:1769074787.467749:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934016 00000100:00100000:2.0:1769074787.467768:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14314 00000100:00100000:2.0:1769074787.467776:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616933504:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:2.0:1769074787.467853:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.467875:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074787.467883:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.467886:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.467889:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.467924:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 22 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074787.467938:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 22 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:2.0:1769074787.467952:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625183616:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074787.468033:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074787.468292:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14315 00000100:00100000:3.0:1769074787.468303:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616933632:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074787.468326:0:28519:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468339:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 16 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074787.468346:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468347:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468350:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468372:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 16 pid: 28954 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074787.468386:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 16 pid: 28954 timeout: 1875 lvb_type: 1 00000100:00100000:3.0:1769074787.468396:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625183744:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074787.468423:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074787.468439:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14316 00000100:00100000:3.0:1769074787.468446:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616933760:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074787.468462:0:28954:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468474:0:28954:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074787.468480:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468481:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468484:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468500:0:28954:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 16 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074787.468509:0:28954:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 16 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:3.0:1769074787.468517:0:28954:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625183872:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074787.468538:0:28954:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6434a89c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074787.468556:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14317 00000100:00100000:3.0:1769074787.468562:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616933888:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074787.468581:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468592:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074787.468598:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468599:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468602:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468619:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 22 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074787.468627:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 22 pid: 8398 timeout: 1875 lvb_type: 1 00000100:00100000:3.0:1769074787.468636:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625184000:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074787.468658:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b543624b40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074787.468678:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14318 00000100:00100000:3.0:1769074787.468686:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616934016:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074787.468704:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468720:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468727:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468728:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.468731:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468751:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 22 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.468759:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 22 pid: 8396 timeout: 1875 lvb_type: 0 00000100:00100000:3.0:1769074787.468767:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625184128:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074787.468792:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f5600 going to sleep for 11 seconds 00000100:00080000:1.0:1769074787.469095:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074787.469101:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074787.471644:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074787.471883:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934144 00000100:00100000:0.0:1769074787.471894:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934144 00000100:00100000:0.0:1769074787.472019:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14319 00000100:00100000:0.0:1769074787.472029:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616934144:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:0.0:1769074787.472086:0:10284:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.472106:0:10284:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 17 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074787.472114:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.472116:0:10284:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.472119:0:10284:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.472164:0:10284:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 17 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074787.472178:0:10284:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 17 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:0.0:1769074787.472192:0:10284:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625184256:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074787.472276:0:10284:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074787.488140:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625184000:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074787.488174:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074787.488263:0:28954:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:28954:1855007625183872:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074787.488278:0:28954:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.488382:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625184128:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074787.488397:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.489374:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cde5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625183744:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074787.489398:0:28519:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.491712:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625183616:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074787.491743:0:8398:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074787.491926:0:10284:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10284:1855007625184256:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074787.491942:0:10284:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.499256:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074787.499406:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934528 00000100:00100000:0.0:1769074787.499417:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934528 00010000:00010000:0.0:1769074787.499461:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 23 pid: 8398 timeout: 1875 lvb_type: 1 00000100:00100000:0.0:1769074787.499539:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7149 00000100:00100000:0.0:1769074787.499549:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616934528:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074787.499575:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074787.499594:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 23 pid: 8398 timeout: 1875 lvb_type: 1 00010000:00010000:0.0:1769074787.499608:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 23 pid: 8398 timeout: 1875 lvb_type: 1 00010000:00010000:0.0:1769074787.499632:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547f0f000/0x7fe2323adfd543a0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1a2 expref: 23 pid: 8398 timeout: 1875 lvb_type: 1 00010000:00010000:0.0:1769074787.499648:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.499651:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.499655:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.499664:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.499700:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074787.499762:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2407:x1855007616934528:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 215us (519us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.499769:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7149 00010000:00010000:3.0:1769074787.499834:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499846:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.499860:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:3.0:1769074787.499877:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499886:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499894:0:28955:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499902:0:28955:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d96000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.499906:0:28955:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499916:0:28955:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499928:0:28955:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.499935:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d96000/0x7fe2323adfd5446b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.499990:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616933888:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 31427us (32955us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.499999:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14317 00000100:00100000:1.0:1769074787.500077:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934272 00000100:00100000:1.0:1769074787.500083:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934272 00000100:00100000:3.0:1769074787.500104:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074787.500109:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 21 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:1.0:1769074787.500139:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934656 00000100:00100000:1.0:1769074787.500153:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934656 00010000:00010000:1.0:1769074787.500167:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 18 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:1.0:1769074787.500192:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7150 00000100:00100000:3.0:1769074787.500198:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.500205:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616934272:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074787.500216:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074787.500231:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 21 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:3.0:1769074787.500231:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00010000:00010000:1.0:1769074787.500245:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 21 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:3.0:1769074787.500251:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934400 00000100:00100000:3.0:1769074787.500257:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934400 00010000:00010000:1.0:1769074787.500260:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd543ae lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c18d expref: 21 pid: 27844 timeout: 1875 lvb_type: 1 00010000:00010000:1.0:1769074787.500271:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.500273:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.500276:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 19 pid: 27844 timeout: 1875 lvb_type: 1 00010000:00010000:1.0:1769074787.500278:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500296:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.500300:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934784 00000100:00100000:3.0:1769074787.500305:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934784 00010000:00010000:3.0:1769074787.500318:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 21 pid: 8396 timeout: 1875 lvb_type: 0 00010000:00010000:1.0:1769074787.500320:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074787.500335:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7151 00000100:00100000:3.0:1769074787.500343:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616934656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074787.500352:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1769074787.500355:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616934272:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 152us (497us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.500364:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7150 00010000:00010000:3.0:1769074787.500368:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 19 pid: 27844 timeout: 1875 lvb_type: 1 00010000:00010000:3.0:1769074787.500380:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 19 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:1.0:1769074787.500387:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7152 00010000:00010000:3.0:1769074787.500399:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e000/0x7fe2323adfd543a7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1b0 expref: 19 pid: 27844 timeout: 1875 lvb_type: 1 00000100:00100000:1.0:1769074787.500400:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616934400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074787.500416:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074787.500425:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.500430:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.500431:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 18 pid: 27844 timeout: 1875 lvb_type: 1 00010000:00010000:3.0:1769074787.500435:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500444:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 18 pid: 27844 timeout: 1875 lvb_type: 1 00010000:00010000:1.0:1769074787.500460:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd543b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c19b expref: 18 pid: 27844 timeout: 1875 lvb_type: 1 00010000:00010000:3.0:1769074787.500473:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500477:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.500480:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.500485:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500493:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500501:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074787.500516:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074787.500524:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616934656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 182us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.500533:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7151 00000100:00100000:1.0:1769074787.500546:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616934400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 147us (348us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.500554:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7152 00000100:00100000:3.0:1769074787.500576:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7153 00000100:00100000:3.0:1769074787.500586:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616934784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074787.500594:0:28954:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500595:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074787.500606:0:28954:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500609:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 20 pid: 8396 timeout: 1875 lvb_type: 0 00010000:00010000:3.0:1769074787.500625:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 20 pid: 8396 timeout: 1875 lvb_type: 0 00010000:00010000:1.0:1769074787.500628:0:28954:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500639:0:28954:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500642:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd543f4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c1ef expref: 20 pid: 8396 timeout: 1875 lvb_type: 0 00010000:00010000:3.0:1769074787.500665:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.500668:0:28954:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500669:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.500674:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500680:0:28954:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074787.500686:0:28954:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500688:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500701:0:28954:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500717:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074787.500738:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616934784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 154us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.500745:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7153 00010000:00010000:3.0:1769074787.500765:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500773:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500786:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500792:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500800:0:27843:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500828:0:27843:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.500832:0:27843:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500840:0:27843:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500853:0:28954:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500859:0:27843:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.500866:0:28954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54464 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.500869:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97800/0x7fe2323adfd54472 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.500926:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2408:x1855007616934016:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 32241us (33869us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.500936:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14318 00000100:00100000:1.0:1769074787.500938:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616933760:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 32487us (33937us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.500950:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14316 00010000:00010000:1.0:1769074787.500993:0:8398:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501004:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501020:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501029:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501038:0:8398:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501045:0:8398:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074787.501049:0:8398:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501057:0:8398:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501080:0:8398:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.501088:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54456 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.501137:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2408:x1855007616933504:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 33361us (34236us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.501154:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14314 00010000:00010000:0.0:1769074787.501419:0:10284:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501429:0:10284:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501443:0:10284:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501451:0:10284:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501458:0:10284:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501464:0:10284:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f70c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074787.501467:0:10284:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501474:0:10284:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501489:0:10284:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.501495:0:10284:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f70c00/0x7fe2323adfd54479 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10284 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074787.501768:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2409:x1855007616934144:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 29739us (30133us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.501777:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14319 00000100:00100000:1.0:1769074787.508937:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.509195:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616934912 00000100:00100000:2.0:1769074787.509206:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616934912 00010000:00010000:2.0:1769074787.509246:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 14 pid: 28954 timeout: 1875 lvb_type: 1 00000100:00100000:2.0:1769074787.509293:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7154 00000100:00100000:2.0:1769074787.509303:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007616934912:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074787.509318:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074787.509339:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 14 pid: 28954 timeout: 1875 lvb_type: 1 00010000:00010000:2.0:1769074787.509359:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 14 pid: 28954 timeout: 1875 lvb_type: 1 00010000:00010000:2.0:1769074787.509382:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0e200/0x7fe2323adfd54392 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c194 expref: 14 pid: 28954 timeout: 1875 lvb_type: 1 00010000:00010000:2.0:1769074787.509398:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.509401:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.509406:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.509415:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.509485:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074787.509551:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2406:x1855007616934912:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 249us (624us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.509560:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7154 00010000:00010000:1.0:1769074787.515595:0:28519:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515621:0:28519:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515655:0:28519:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515664:0:28519:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515673:0:28519:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515680:0:28519:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d96e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074787.515685:0:28519:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515694:0:28519:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515714:0:28519:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.515721:0:28519:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd5445d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.516132:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007616933632:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 47823us (49167us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.516147:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14315 00000100:00100000:0.0:1769074787.530184:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.530417:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935168 00000100:00100000:2.0:1769074787.530429:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935168 00000100:00100000:2.0:1769074787.530544:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13365 00000100:00100000:2.0:1769074787.530556:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63053:x1855007616935168:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074787.530616:0:8855:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1769074787.530644:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0fa00/0x7fe2323adfd54368 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c171 expref: 225 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.530672:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5443a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c20b expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.530682:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.530688:0:8855:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074787.530717:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.530730:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.530738:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000040:00080000:2.0:1769074787.531038:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2438, 88 off208760 00000004:00080000:2.0:1769074787.531047:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775373239934ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.531103:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2435, 88 off208232 00000004:00080000:2.0:1769074787.531111:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775373303639ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.531156:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2436, 88 off208320 00000004:00080000:2.0:1769074787.531162:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775373354837ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.531182:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2439, 88 off208848 00000004:00080000:2.0:1769074787.531187:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775373380019ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.531202:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2440, 88 off208936 00000004:00080000:2.0:1769074787.531206:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775373398689ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.531215:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2437, 88 off208408 00000004:00080000:2.0:1769074787.531220:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775373412668ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.531238:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2438, 88 off208496 00000004:00080000:2.0:1769074787.531243:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775373435652ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.531258:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2439, 88 off208584 00000004:00080000:2.0:1769074787.531262:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775373454791ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.531277:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2441, 88 off209024 00000004:00080000:2.0:1769074787.531504:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775373696862ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.531565:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2440, 88 off208672 00000004:00080000:2.0:1769074787.531571:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775373763781ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.531594:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2442, 88 off209112 00000004:00080000:2.0:1769074787.531599:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775373791819ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.533891:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2441, 88 off208760 00000004:00080000:2.0:1769074787.533910:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775376102990ns, next at 1706953247299ns, rc = 0 00000040:00080000:2.0:1769074787.533935:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2443, 88 off209200 00000004:00080000:2.0:1769074787.533940:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775376133030ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.534094:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2444, 88 off209288 00000004:00080000:2.0:1769074787.534100:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775376292610ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.534389:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2445, 88 off209376 00000004:00080000:2.0:1769074787.534395:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775376587753ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.534492:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2446, 88 off209464 00000004:00080000:2.0:1769074787.534498:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1775376690790ns, next at 1706951295843ns, rc = 0 00000040:00080000:2.0:1769074787.534745:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2442, 88 off208848 00000004:00080000:2.0:1769074787.534751:0:8855:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1775376944091ns, next at 1706953247299ns, rc = 0 00000001:00080000:3.0:1769074787.537210:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944009 is committed 00000004:00080000:3.0:1769074787.537221:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1946 committed 00000004:00080000:3.0:1769074787.537293:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1949 committed 00000004:00080000:3.0:1769074787.537308:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1948 committed 00000004:00080000:3.0:1769074787.537311:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1947 committed 00000004:00080000:3.0:1769074787.537313:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1946 committed 00000004:00080000:3.0:1769074787.537315:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1945 committed 00000100:00100000:1.0:1769074787.537511:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625184512:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.537589:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074787.537622:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185024:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.537645:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074787.537668:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1769074787.537674:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625184640:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.537694:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074787.537708:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185152:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.537725:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074787.537753:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625184512 00000100:00100000:1.0:1769074787.537761:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625184512 00000100:00100000:1.0:1769074787.537803:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185024 00000100:00100000:1.0:1769074787.537836:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185024 00000100:00100000:1.0:1769074787.537856:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625184640 00000100:00100000:1.0:1769074787.537860:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625184640 00000100:00100000:2.0:1769074787.538267:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625184384:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538313:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.538335:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625184896:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538354:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.538372:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1769074787.538379:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625184768:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538399:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.538429:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625185280:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538450:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074787.538478:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185152 00000100:00100000:2.0:1769074787.538483:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185152 00000100:00100000:2.0:1769074787.538507:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625184384 00000100:00100000:2.0:1769074787.538510:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625184384 00000100:00100000:2.0:1769074787.538519:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625184896 00000100:00100000:2.0:1769074787.538521:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625184896 00000100:00100000:2.0:1769074787.538530:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625184768 00000100:00100000:2.0:1769074787.538533:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625184768 00000100:00100000:2.0:1769074787.538540:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185280 00000100:00100000:2.0:1769074787.538543:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185280 00000100:00100000:2.0:1769074787.538556:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14320 00000100:00100000:2.0:1769074787.538564:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625184512:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538765:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625184512:12345-0@lo:2:osp-syn-1-0.0 Request processed in 200us (1175us total) trans 17179874569 rc 0/0 00000100:00100000:2.0:1769074787.538774:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14320 00000100:00100000:2.0:1769074787.538795:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14321 00000100:00100000:2.0:1769074787.538802:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625185024:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538903:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625185024:12345-0@lo:2:osp-syn-1-0.0 Request processed in 101us (1257us total) trans 17179874570 rc 0/0 00000100:00100000:2.0:1769074787.538910:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14321 00000100:00100000:2.0:1769074787.538923:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14322 00000100:00100000:2.0:1769074787.538929:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625185152:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.538994:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625185152:12345-0@lo:2:osp-syn-1-0.0 Request processed in 65us (1267us total) trans 17179874571 rc 0/0 00000100:00100000:2.0:1769074787.539003:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14322 00000100:00100000:2.0:1769074787.539017:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14323 00000100:00100000:2.0:1769074787.539022:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625184384:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.539110:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625184384:12345-0@lo:2:osp-syn-1-0.0 Request processed in 87us (796us total) trans 17179874572 rc 0/0 00000100:00100000:2.0:1769074787.539117:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14323 00000100:00100000:2.0:1769074787.539129:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14324 00000100:00100000:2.0:1769074787.539134:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625184896:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.539199:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625184896:12345-0@lo:2:osp-syn-1-0.0 Request processed in 64us (843us total) trans 17179874573 rc 0/0 00000100:00100000:2.0:1769074787.539206:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14324 00000100:00100000:2.0:1769074787.539217:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14325 00000100:00100000:2.0:1769074787.539222:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625184768:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.539283:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625184768:12345-0@lo:2:osp-syn-1-0.0 Request processed in 60us (882us total) trans 17179874574 rc 0/0 00000100:00100000:2.0:1769074787.539289:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14325 00000100:00100000:2.0:1769074787.539299:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14326 00000100:00100000:2.0:1769074787.539305:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+11:3652:x1855007625185280:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.539367:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+12:3652:x1855007625185280:12345-0@lo:2:osp-syn-1-0.0 Request processed in 61us (915us total) trans 17179874575 rc 0/0 00000100:00100000:2.0:1769074787.539374:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14326 00000004:00080000:2.0:1769074787.539423:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dea00/2, rc 0, transno 17179874571 00000100:00100000:2.0:1769074787.539435:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185152:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074787.539446:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475de300/2, rc 0, transno 17179874569 00000100:00100000:2.0:1769074787.539450:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625184512:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074787.539458:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ded80/2, rc 0, transno 17179874570 00000100:00100000:2.0:1769074787.539461:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185024:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074787.539484:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc000/2, rc 0, transno 17179874574 00000100:00100000:2.0:1769074787.539488:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625184768:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074787.539495:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dc700/2, rc 0, transno 17179874575 00000100:00100000:2.0:1769074787.539498:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625185280:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074787.539506:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dca80/2, rc 0, transno 17179874572 00000100:00100000:2.0:1769074787.539510:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625184384:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074787.539517:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475df480/2, rc 0, transno 17179874573 00000100:00100000:2.0:1769074787.539520:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625184896:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.540156:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14327 00000100:00100000:1.0:1769074787.540164:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625184640:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.540276:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625184640:12345-0@lo:2:osp-syn-1-0.0 Request processed in 113us (2580us total) trans 17179874576 rc 0/0 00000100:00100000:1.0:1769074787.540298:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14327 00000004:00080000:2.0:1769074787.540866:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddf80/2, rc 0, transno 17179874576 00000100:00100000:2.0:1769074787.540873:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625184640:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.540896:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542faad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625185408:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074787.540934:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074787.541007:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185408 00000100:00100000:1.0:1769074787.541014:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185408 00000100:00100000:0.0:1769074787.541221:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625185792:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.541250:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1769074787.541323:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1945 committed 00000004:00080000:3.0:1769074787.541327:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1944 committed 00000004:00080000:3.0:1769074787.541329:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1944 committed 00000004:00080000:3.0:1769074787.541331:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1943 committed 00000004:00080000:3.0:1769074787.541333:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1942 committed 00000004:00080000:3.0:1769074787.541335:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1941 committed 00000004:00080000:3.0:1769074787.541336:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1943 committed 00000004:00080000:3.0:1769074787.541338:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1942 committed 00000004:00080000:3.0:1769074787.541340:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1940 committed 00000004:00080000:3.0:1769074787.541342:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1939 committed 00000004:00080000:3.0:1769074787.541344:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1941 committed 00000001:00100000:3.0:1769074787.541362:0:32377:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000001:00080000:3.0:1769074787.541371:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944006 is committed 00000020:00000080:3.0:1769074787.541375:0:32377:0:(genops.c:904:class_export_put()) final put ffff89b67b10c000/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000020:00000080:3.0:1769074787.541428:0:35568:0:(genops.c:863:class_export_destroy()) destroying export ffff89b67b10c000/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b for lustre-MDT0000 00000020:00000080:3.0:1769074787.541485:0:35568:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b67b10c800 for lustre-MDT0000 00000100:00100000:0.0:1769074787.541763:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625186304:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.541789:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074787.541867:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185792 00000100:00100000:1.0:1769074787.541873:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185792 00000100:00100000:1.0:1769074787.541903:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14328 00000100:00100000:1.0:1769074787.541911:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+13:3652:x1855007625185408:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074787.542047:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625185408:12345-0@lo:2:osp-syn-1-0.0 Request processed in 136us (1112us total) trans 17179874577 rc 0/0 00000100:00100000:1.0:1769074787.542054:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14328 00000100:00100000:1.0:1769074787.542081:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14329 00000100:00100000:1.0:1769074787.542087:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625185792:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074787.542168:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625185792:12345-0@lo:2:osp-syn-0-0.0 Request processed in 80us (916us total) trans 21474841240 rc 0/0 00000100:00100000:1.0:1769074787.542174:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14329 00000004:00010000:2.0:1769074787.542372:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000004c990169 reply state 000000007457093a transno 8589944009 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542396:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542495:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.542498:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:2.0:1769074787.542502:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00080000:2.0:1769074787.542635:0:8855:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1769074787.542651:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63053:x1855007616935168:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 12096us (12478us total) trans 8589944009 rc 0/0 00000100:00100000:2.0:1769074787.542660:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13365 00010000:00010000:2.0:1769074787.542682:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542689:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542696:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542701:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074787.542706:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542713:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542723:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.542725:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074787.542729:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.542734:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54480 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000004:00080000:0.0:1769074787.548905:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2aa00/2, rc 0, transno 21474841240 00000100:00100000:0.0:1769074787.548921:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625185792:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.548929:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1769074787.548940:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625185920:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.548994:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074787.549022:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625186432:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.549041:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074787.549090:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625186304 00000100:00100000:0.0:1769074787.549097:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625186304 00000100:00100000:0.0:1769074787.549140:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185920 00000100:00100000:0.0:1769074787.549144:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185920 00000100:00100000:0.0:1769074787.549159:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625186432 00000100:00100000:0.0:1769074787.549162:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625186432 00000100:00100000:0.0:1769074787.549181:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14330 00000100:00100000:0.0:1769074787.549189:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625186304:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074787.549403:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14331 00000100:00100000:1.0:1769074787.549514:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625185920:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.550717:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625186304:12345-0@lo:2:osp-syn-0-0.0 Request processed in 1522us (8920us total) trans 21474841241 rc 0/0 00000100:00100000:0.0:1769074787.550728:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14330 00000100:00100000:0.0:1769074787.550756:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14332 00000100:00100000:0.0:1769074787.550764:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625186432:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.550867:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625186432:12345-0@lo:2:osp-syn-0-0.0 Request processed in 104us (1824us total) trans 21474841242 rc 0/0 00000100:00100000:0.0:1769074787.550873:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14332 00000004:00080000:0.0:1769074787.550937:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f28380/2, rc 0, transno 21474841242 00000100:00100000:0.0:1769074787.550943:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625186432:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074787.550952:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2b100/2, rc 0, transno 21474841241 00000100:00100000:0.0:1769074787.550955:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625186304:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.550980:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185536:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.551011:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074787.551033:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625186048:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.551050:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074787.551082:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1769074787.551090:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185664:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.551109:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074787.551124:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625186176:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.551146:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074787.551170:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185536 00000100:00100000:0.0:1769074787.551177:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185536 00000100:00100000:0.0:1769074787.551202:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625186048 00000100:00100000:0.0:1769074787.551205:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625186048 00000100:00100000:0.0:1769074787.551219:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625185664 00000100:00100000:0.0:1769074787.551222:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625185664 00000100:00100000:0.0:1769074787.551233:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625186176 00000100:00100000:0.0:1769074787.551235:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625186176 00000100:00100000:0.0:1769074787.551246:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14333 00000100:00100000:0.0:1769074787.551251:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625185536:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.551334:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625185536:12345-0@lo:2:osp-syn-0-0.0 Request processed in 81us (321us total) trans 21474841243 rc 0/0 00000100:00100000:0.0:1769074787.551340:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14333 00000100:00100000:0.0:1769074787.551352:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14334 00000100:00100000:0.0:1769074787.551358:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625186048:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074787.551597:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625185920:12345-0@lo:2:osp-syn-0-0.0 Request processed in 2086us (2601us total) trans 21474841245 rc 0/0 00000100:00100000:1.0:1769074787.551604:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14331 00000100:00100000:1.0:1769074787.551618:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14335 00000100:00100000:1.0:1769074787.551624:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625185664:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074787.551691:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625185664:12345-0@lo:2:osp-syn-0-0.0 Request processed in 67us (580us total) trans 21474841246 rc 0/0 00000100:00100000:1.0:1769074787.551697:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14335 00000100:00100000:1.0:1769074787.551708:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14336 00000100:00100000:1.0:1769074787.551714:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625186176:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074787.551771:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625186176:12345-0@lo:2:osp-syn-0-0.0 Request processed in 57us (624us total) trans 21474841247 rc 0/0 00000100:00100000:1.0:1769074787.551777:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14336 00000004:00080000:1.0:1769074787.551803:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2bb80/2, rc 0, transno 21474841245 00000100:00100000:1.0:1769074787.551822:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625185920:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074787.554240:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625186048:12345-0@lo:2:osp-syn-0-0.0 Request processed in 2869us (3176us total) trans 21474841244 rc 0/0 00000100:00100000:0.0:1769074787.554256:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14334 00000004:00080000:0.0:1769074787.554348:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2a300/2, rc 0, transno 21474841246 00000100:00100000:0.0:1769074787.554355:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185664:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074787.554366:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2ad80/2, rc 0, transno 21474841247 00000100:00100000:0.0:1769074787.554369:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625186176:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074787.554378:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f28000/2, rc 0, transno 21474841243 00000100:00100000:0.0:1769074787.554381:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625185536:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074787.554388:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f29180/2, rc 0, transno 21474841244 00000100:00100000:0.0:1769074787.554392:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625186048:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074787.554422:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b542faad80/2, rc 0, transno 17179874577 00000100:00100000:0.0:1769074787.554427:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542faad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625185408:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074787.555826:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.555912:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935296 00000100:00100000:2.0:1769074787.555921:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935296 00000100:00100000:2.0:1769074787.556010:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13366 00000100:00100000:2.0:1769074787.556020:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:63053:x1855007616935296:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.556045:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074787.556079:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c219 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.556136:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 222 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.556151:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 222 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.556170:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.556173:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074787.556176:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba400, rc 0) 00000100:00100000:2.0:1769074787.556234:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63053:x1855007616935296:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 216us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.556241:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13366 00000100:00100000:1.0:1769074787.565495:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.566191:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935424 00000100:00100000:2.0:1769074787.566203:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935424 00000100:00100000:2.0:1769074787.566264:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13367 00000100:00100000:2.0:1769074787.566274:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63053:x1855007616935424:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.566316:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074787.566567:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5448e lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c220 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.566674:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54495 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.566689:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54495 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.566701:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54495 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074787.566736:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:2.0:1769074787.566741:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074787.566759:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54495 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074787.566784:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5448e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c220 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.566801:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54495 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c220 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.566834:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074787.566837:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074787.566842:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:2.0:1769074787.566930:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616935424:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 657us (1444us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.566938:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13367 00000100:00100000:1.0:1769074787.579494:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.579534:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.579563:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.579595:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.579617:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935808 00000100:00100000:2.0:1769074787.579618:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935552 00000100:00100000:1.0:1769074787.579627:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935808 00000100:00100000:2.0:1769074787.579627:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935552 00000100:00100000:2.0:1769074787.579669:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936064 00000100:00100000:2.0:1769074787.579672:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936064 00000100:00100000:1.0:1769074787.583689:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935680 00000100:00100000:1.0:1769074787.583705:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935680 00000100:00100000:2.0:1769074787.583711:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14337 00000100:00100000:2.0:1769074787.583726:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616935552:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.583747:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1769074787.583757:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14338 00000100:00100000:1.0:1769074787.583764:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2409:x1855007616935808:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.583770:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c227 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.583771:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074787.583782:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c235 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074787.583796:0:9048:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bb:0x0] initial LVB size: 2097152, mtime: 0x6971f058, atime: 0x6971f058, ctime: 0x6971f057, blocks: 0x800 00002000:00010000:2.0:1769074787.583796:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bc:0x0] initial LVB size: 1048576, mtime: 0x6971f054, atime: 0x6971f052, ctime: 0x6971f054, blocks: 0x7e8 00010000:00010000:1.0:1769074787.583827:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.583827:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.583830:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.583830:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.583840:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074787.583840:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 19 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.583857:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.583870:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053e00, rc 0) 00000100:00100000:1.0:1769074787.583936:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2409:x1855007616935808:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 171us (4401us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.583947:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14338 00000100:00100000:1.0:1769074787.583968:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14339 00000100:00100000:1.0:1769074787.583977:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616936064:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074787.583983:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074787.583992:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c243 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074787.584004:0:9048:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bf:0x0] initial LVB size: 4805728, mtime: 0x6971f05c, atime: 0x6971f059, ctime: 0x6971f05c, blocks: 0x14b0 00010000:00010000:1.0:1769074787.584012:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.584013:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.584018:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 20 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.584028:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 20 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.584035:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052e00, rc 0) 00000100:00100000:1.0:1769074787.584056:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616936064:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 81us (4492us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.584074:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14339 00000100:00100000:1.0:1769074787.584090:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14340 00000100:00100000:1.0:1769074787.584096:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2409:x1855007616935680:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074787.584103:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074787.584111:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c22e expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074787.584128:0:9048:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2ba:0x0] initial LVB size: 2097152, mtime: 0x6971f058, atime: 0x6971f058, ctime: 0x6971f058, blocks: 0x800 00010000:00010000:1.0:1769074787.584136:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.584138:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.584142:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.584152:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 14 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.584162:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052000, rc 0) 00000100:00100000:1.0:1769074787.584182:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2409:x1855007616935680:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 86us (4587us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.584188:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14340 00010000:00010000:2.0:1769074787.584206:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 19 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074787.584216:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:2.0:1769074787.584237:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616935552:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 518us (4754us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.584242:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14337 00000100:00100000:1.0:1769074787.584255:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.584333:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.584350:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616935936 00000100:00100000:1.0:1769074787.584356:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616935936 00000100:00100000:1.0:1769074787.584376:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936192 00000100:00100000:1.0:1769074787.584379:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936192 00000100:00100000:1.0:1769074787.584396:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14341 00000100:00100000:1.0:1769074787.584402:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007616935936:12345-192.168.204.60@tcp:101:lfs.0 00000100:00100000:2.0:1769074787.584633:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14342 00000100:00100000:2.0:1769074787.584642:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2409:x1855007616936192:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074787.584650:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074787.584663:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c24a expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074787.584677:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bc:0x0] initial LVB size: 4194304, mtime: 0x6971f059, atime: 0x6971f058, ctime: 0x6971f059, blocks: 0x1008 00010000:00010000:2.0:1769074787.584686:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.584688:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.584693:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 15 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074787.584702:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 15 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074787.584715:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602b800, rc 0) 00000100:00100000:2.0:1769074787.584735:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616936192:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 96us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.584741:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14342 00010000:00010000:1.0:1769074787.585155:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074787.585171:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c23c expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074787.585186:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2be:0x0] initial LVB size: 2097152, mtime: 0x6971f058, atime: 0x6971f058, ctime: 0x6971f058, blocks: 0x808 00010000:00010000:1.0:1769074787.585197:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.585198:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.585205:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.585217:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 20 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074787.585226:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053c00, rc 0) 00000100:00100000:1.0:1769074787.585253:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616935936:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 850us (997us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.585260:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14341 00000100:00100000:1.0:1769074787.685976:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.686185:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936320 00000100:00100000:3.0:1769074787.686202:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936320 00000100:00100000:3.0:1769074787.686299:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14343 00000100:00100000:3.0:1769074787.686317:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616936320:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:3.0:1769074787.686742:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616936320:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 425us (773us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074787.686751:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14343 00000100:00100000:1.0:1769074787.700860:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.700921:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.701014:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936448 00000100:00100000:2.0:1769074787.701026:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936448 00000100:00100000:2.0:1769074787.701088:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936704 00000100:00100000:2.0:1769074787.701092:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936704 00000100:00100000:2.0:1769074787.701116:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14344 00000100:00100000:2.0:1769074787.701124:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616936448:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:2.0:1769074787.701230:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616936448:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 107us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.701238:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14344 00000100:00100000:2.0:1769074787.701255:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14345 00000100:00100000:2.0:1769074787.701261:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616936704:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:2.0:1769074787.701294:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616936704:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 34us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074787.701299:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14345 00000100:00100000:0.0:1769074787.708877:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.709157:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936576 00000100:00100000:1.0:1769074787.709171:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936576 00000100:00100000:1.0:1769074787.709237:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14346 00000100:00100000:1.0:1769074787.709248:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616936576:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:1.0:1769074787.709388:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cde4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616936576:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 141us (521us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.709397:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14346 00000100:00100000:1.0:1769074787.724670:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.724717:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.724751:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.724780:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.724847:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936832 00000100:00100000:1.0:1769074787.724858:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936832 00000100:00100000:1.0:1769074787.724906:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616936960 00000100:00100000:1.0:1769074787.724909:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616936960 00000100:00100000:1.0:1769074787.724924:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937216 00000100:00100000:1.0:1769074787.724926:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937216 00000100:00100000:1.0:1769074787.724940:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937088 00000100:00100000:1.0:1769074787.724943:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937088 00000100:00100000:1.0:1769074787.724965:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14347 00000100:00100000:1.0:1769074787.724975:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616936832:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:1.0:1769074787.725098:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616936832:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 124us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.725106:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14347 00000100:00100000:1.0:1769074787.725124:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14348 00000100:00100000:1.0:1769074787.725131:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616936960:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074787.725232:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14349 00000100:00100000:0.0:1769074787.725242:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616937216:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074787.725304:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616937216:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 63us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.725311:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14349 00000100:00100000:0.0:1769074787.725325:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14350 00000100:00100000:0.0:1769074787.725332:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616937088:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074787.725370:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616937088:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 39us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074787.725376:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14350 00000100:00100000:1.0:1769074787.725709:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616936960:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 576us (990us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074787.725717:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14348 00010000:00010000:0.0:1769074787.741072:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074787.750123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.750278:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.753734:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074787.756741:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937472 00000100:00100000:1.0:1769074787.756758:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937472 00000100:00100000:1.0:1769074787.756857:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937344 00000100:00100000:1.0:1769074787.756862:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937344 00000100:00100000:1.0:1769074787.756898:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8696 00000100:00100000:1.0:1769074787.756908:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616937472:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074787.756946:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.756962:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.756964:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074787.756969:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.756982:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074787.757588:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8697 00000100:00100000:0.0:1769074787.757602:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616937344:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074787.757623:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.757632:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.757634:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.757637:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.757653:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.762961:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.762992:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.763012:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.763068:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937600 00000100:00100000:2.0:1769074787.763080:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937600 00000100:00100000:2.0:1769074787.763155:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937728 00000100:00100000:2.0:1769074787.763158:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937728 00000100:00100000:2.0:1769074787.763182:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937856 00000100:00100000:2.0:1769074787.763185:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937856 00000100:00100000:2.0:1769074787.763203:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616937984 00000100:00100000:2.0:1769074787.763206:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616937984 00000100:00100000:2.0:1769074787.763228:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8698 00000100:00100000:2.0:1769074787.763239:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616937600:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074787.763281:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.763296:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.763298:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.763304:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.763318:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.763908:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8699 00000100:00100000:2.0:1769074787.763918:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616937728:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074787.763939:0:28444:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.763948:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.763950:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.763954:0:28444:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.763970:0:28444:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.764481:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8700 00000100:00100000:2.0:1769074787.764491:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616937856:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074787.764511:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.764520:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.764522:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.764526:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.764535:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.770035:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8701 00000100:00100000:2.0:1769074787.770069:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007616937984:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074787.770112:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.770129:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.770131:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.770138:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.770166:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.772551:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.773977:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938112 00000100:00100000:2.0:1769074787.773993:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938112 00000100:00100000:2.0:1769074787.778877:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8702 00000100:00100000:2.0:1769074787.778898:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616938112:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074787.778938:0:24028:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.778953:0:24028:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.778956:0:24028:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.778961:0:24028:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.778974:0:24028:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.794959:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.794977:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.794986:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.794994:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547f71400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074787.794998:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.795006:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.795024:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.795031:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f71400/0x7fe2323adfd544cd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074787.795153:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616937344:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 37539us (44860us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074787.795168:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8697 00010000:00010000:1.0:1769074787.813763:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.813783:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.813793:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.813801:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b052a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074787.813850:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.813860:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.813881:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.813889:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd544c6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.813984:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007616937472:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 57070us (63865us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074787.813995:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8696 00010000:00010000:1.0:1769074787.819113:0:28444:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.819135:0:28444:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.819144:0:28444:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.819152:0:28444:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074787.819156:0:28444:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.819164:0:28444:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.819186:0:28444:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074787.819194:0:28444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8a00/0x7fe2323adfd544db lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.819296:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616937728:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 55374us (56342us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074787.819308:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8699 00010000:00010000:3.0:1769074787.819411:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.819423:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.819431:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.819438:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.819442:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.819449:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.819467:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.819473:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9000/0x7fe2323adfd544d4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.819516:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616937600:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 56278us (65791us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074787.819524:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8698 00010000:00010000:3.0:1769074787.847192:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.847212:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.847221:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.847228:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.847233:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.847241:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.847261:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.847273:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c8800/0x7fe2323adfd544e2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.847448:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616937856:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 82952us (84448us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074787.847460:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8700 00010000:00010000:2.0:1769074787.851372:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.851394:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.851404:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.851411:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074787.851416:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.851424:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.851444:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.851450:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8e00/0x7fe2323adfd544e9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.851546:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616937984:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 81491us (88527us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074787.851557:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8701 00010000:00010000:3.0:1769074787.854769:0:24028:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.854788:0:24028:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.854798:0:24028:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.854823:0:24028:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b6725b5800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.854827:0:24028:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.854836:0:24028:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.854854:0:24028:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.854862:0:24028:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b6725b5800/0x7fe2323adfd544f0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.855023:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007616938112:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 76127us (82475us total) trans 0 rc 611424/611424 00000100:00100000:2.0:1769074787.855035:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8702 00000100:00100000:1.0:1769074787.879664:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074787.879846:0:24028:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938240 00000100:00100000:2.0:1769074787.879857:0:24028:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938240 00000100:00100000:2.0:1769074787.879943:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8703 00000100:00100000:2.0:1769074787.879952:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616938240:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:2.0:1769074787.880003:0:24028:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bd:0x0] initial LVB size: 3145728, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1000 00010000:00010000:2.0:1769074787.880013:0:24028:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.880027:0:24028:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.880030:0:24028:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.880035:0:24028:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.880049:0:24028:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.886067:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.886111:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.886141:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938368 00000100:00100000:3.0:1769074787.886151:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938368 00000100:00100000:3.0:1769074787.886222:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938496 00000100:00100000:3.0:1769074787.886225:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938496 00000100:00100000:3.0:1769074787.886254:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8704 00000100:00100000:3.0:1769074787.886265:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616938368:12345-192.168.204.60@tcp:4:lfs.0 00000100:00100000:2.0:1769074787.886275:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8705 00000100:00100000:2.0:1769074787.886291:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616938496:12345-192.168.204.60@tcp:4:lfs.0 00010000:00010000:2.0:1769074787.886323:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074787.886338:0:28448:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c0:0x0] initial LVB size: 3145728, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1000 00010000:00010000:2.0:1769074787.886340:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074787.886345:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.886349:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.886352:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.886366:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.886368:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.886371:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.886396:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.886396:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074787.906556:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.906895:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938624 00000100:00100000:3.0:1769074787.906907:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938624 00000100:00100000:3.0:1769074787.906974:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8706 00000100:00100000:3.0:1769074787.906984:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616938624:12345-192.168.204.60@tcp:4:lfs.0 00010000:00010000:3.0:1769074787.907015:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.907036:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.907038:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.907043:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.907066:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000001:00080000:2.0:1769074787.917288:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841248 is committed 00000001:00080000:2.0:1769074787.917304:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841247 is committed 00000001:00100000:2.0:1769074787.917317:0:24714:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000020:00000080:2.0:1769074787.917321:0:24714:0:(genops.c:904:class_export_put()) final put ffff89b5452d6800/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00002000:00010000:3.0:1769074787.918802:0:24028:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:3.0:1769074787.918851:0:24028:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:3.0:1769074787.918868:0:24028:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.918892:0:24028:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.918902:0:24028:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.918910:0:24028:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b718000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.918915:0:24028:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.918925:0:24028:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.918946:0:24028:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.918955:0:24028:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718000/0x7fe2323adfd544f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24028 timeout: 0 lvb_type: 0 00000020:00000080:0.0:1769074787.919175:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b5452d6800/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b for lustre-OST0000 00000020:00000080:1.0:1769074787.919333:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b543618800 for lustre-OST0000 00000001:00080000:0.0:1769074787.923136:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874578 is committed 00000001:00080000:0.0:1769074787.923156:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874577 is committed 00000001:00100000:0.0:1769074787.923173:0:26036:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000020:00000080:0.0:1769074787.923178:0:26036:0:(genops.c:904:class_export_put()) final put ffff89b54361f000/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000020:00000080:1.0:1769074787.925182:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b54361f000/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b for lustre-OST0001 00002000:00010000:2.0:1769074787.928693:0:28448:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:2.0:1769074787.928702:0:28448:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:2.0:1769074787.928719:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.928732:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.928740:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.928747:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074787.928752:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.928759:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.928777:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.928784:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54505 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000020:00000080:1.0:1769074787.929153:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b54361d000 for lustre-OST0001 00000100:00100000:2.0:1769074787.930009:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616938368:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 43738us (43957us total) trans 17179874578 rc 0/0 00000100:00100000:2.0:1769074787.930021:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8704 00000100:00100000:3.0:1769074787.933209:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616938240:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 53242us (53543us total) trans 21474841248 rc 0/0 00000100:00100000:3.0:1769074787.933226:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8703 00000001:00080000:2.0:1769074787.937630:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841249 is committed 00010000:00010000:0.0:1769074787.937960:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.937975:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.937983:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.937990:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54b718600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074787.937995:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.938003:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.938022:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.938029:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd544fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074787.938142:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616938496:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 51852us (52024us total) trans 21474841249 rc 0/0 00000100:00100000:0.0:1769074787.938153:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8705 00000001:00080000:0.0:1769074787.944219:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874579 is committed 00010000:00010000:3.0:1769074787.944487:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.944501:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.944508:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.944513:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074787.944516:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.944522:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.944545:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.944550:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd5450c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074787.944643:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616938624:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 37655us (38091us total) trans 17179874579 rc 0/0 00000100:00100000:3.0:1769074787.944654:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8706 00000100:00100000:2.0:1769074787.955296:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074787.955391:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938880 00000100:00100000:3.0:1769074787.955402:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938880 00000100:00100000:3.0:1769074787.955478:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8707 00000100:00100000:3.0:1769074787.955488:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616938880:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:3.0:1769074787.955546:0:16436:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c1:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x800 00010000:00010000:3.0:1769074787.955556:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.955571:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.955573:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074787.955578:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074787.955592:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.959224:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074787.959308:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616938752 00000100:00100000:0.0:1769074787.959318:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616938752 00000100:00100000:0.0:1769074787.959391:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8708 00000100:00100000:0.0:1769074787.959406:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616938752:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:0.0:1769074787.959453:0:8403:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2be:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x800 00010000:00010000:0.0:1769074787.959461:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.959475:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.959477:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074787.959482:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074787.959495:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074787.981220:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874580 is committed 00002000:00010000:2.0:1769074787.981719:0:16436:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:2.0:1769074787.981724:0:16436:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:2.0:1769074787.981738:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.981754:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.981762:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.981769:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074787.981774:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.981783:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.981801:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074787.981840:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd54513 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074787.981945:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616938880:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 26454us (26654us total) trans 17179874580 rc 0/0 00000100:00100000:2.0:1769074787.981957:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8707 00000001:00080000:2.0:1769074788.004208:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841250 is committed 00002000:00010000:0.0:1769074788.005159:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:0.0:1769074788.005166:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:0.0:1769074788.005180:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.005192:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.005202:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.005209:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074788.005213:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.005233:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.005252:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.005259:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5451a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.005359:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007616938752:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 45956us (46139us total) trans 21474841250 rc 0/0 00000100:00100000:0.0:1769074788.005376:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8708 00000100:00100000:2.0:1769074788.015342:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.015519:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939008 00000100:00100000:0.0:1769074788.015541:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939008 00000100:00100000:0.0:1769074788.015653:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8709 00000100:00100000:0.0:1769074788.015663:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616939008:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:0.0:1769074788.015710:0:8403:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bf:0x0] initial LVB size: 4805728, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x4b0 00010000:00010000:0.0:1769074788.015719:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.015734:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.015737:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.015742:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.015756:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074788.027587:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841251 is committed 00002000:00010000:0.0:1769074788.028117:0:8403:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4808704 00002000:00010000:0.0:1769074788.028123:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:0.0:1769074788.028126:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:0.0:1769074788.028140:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.028152:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.028162:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.028170:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074788.028175:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.028184:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.028204:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.028213:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd54521 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.028316:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616939008:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 12650us (12980us total) trans 21474841251 rc 0/0 00000100:00100000:0.0:1769074788.028328:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8709 00000100:00100000:3.0:1769074788.044344:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.044435:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939136 00000100:00100000:0.0:1769074788.044448:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939136 00000100:00100000:0.0:1769074788.044543:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8710 00000100:00100000:0.0:1769074788.044555:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616939136:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:0.0:1769074788.044616:0:8403:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c3:0x0] initial LVB size: 8388608, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x2000 00010000:00010000:0.0:1769074788.044627:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.044642:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.044644:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.044650:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.044665:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074788.091836:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874581 is committed 00002000:00010000:0.0:1769074788.092437:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:0.0:1769074788.092442:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:0.0:1769074788.092461:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.092478:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.092487:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.092495:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074788.092500:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.092509:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.092540:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.092548:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd54528 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.092679:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007616939136:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 48121us (48339us total) trans 17179874581 rc 0/0 00000100:00100000:0.0:1769074788.092690:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8710 00000100:00100000:3.0:1769074788.120309:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.120491:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939264 00000100:00100000:0.0:1769074788.120502:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939264 00000100:00100000:0.0:1769074788.120587:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8711 00000100:00100000:0.0:1769074788.120596:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616939264:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:0.0:1769074788.120649:0:8403:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c4:0x0] initial LVB size: 4805728, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x4b0 00010000:00010000:0.0:1769074788.120659:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.120674:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.120676:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.120681:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.120695:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074788.130295:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874582 is committed 00002000:00010000:0.0:1769074788.137220:0:8403:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4808704 00002000:00010000:0.0:1769074788.137229:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:0.0:1769074788.137232:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00010000:00010000:0.0:1769074788.137245:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.137258:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.137267:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.137274:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d97400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074788.137278:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.137291:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.137311:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.137319:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547d97400/0x7fe2323adfd5452f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.137447:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616939264:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 16847us (17143us total) trans 17179874582 rc 0/0 00000100:00100000:0.0:1769074788.137458:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8711 00000100:00100000:1.0:1769074788.167720:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.167837:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939520 00000100:00100000:3.0:1769074788.167860:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939520 00000100:00100000:3.0:1769074788.167924:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13368 00000100:00100000:3.0:1769074788.167935:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63053:x1855007616939520:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074788.168002:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074788.168032:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54495 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c220 expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.168061:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.168067:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074788.168257:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.168271:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.168279:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000001:00080000:3.0:1769074788.168322:0:6512:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944010 00000004:00010000:3.0:1769074788.168346:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008c047d79 reply state 0000000099d9c6aa transno 8589944011 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.168357:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.168363:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.168366:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074788.168369:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074788.168475:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63053:x1855007616939520:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 542us (765us total) trans 8589944011 rc 0/0 00000100:00100000:3.0:1769074788.168483:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13368 00000020:00080000:3.0:1769074788.168488:0:6512:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1769074788.172349:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.172463:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939648 00000100:00100000:0.0:1769074788.172474:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939648 00000100:00100000:0.0:1769074788.172534:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14351 00000100:00100000:0.0:1769074788.172545:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:63053:x1855007616939648:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074788.172562:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074788.172582:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c251 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074788.172606:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2b9:0x0] initial LVB size: 2097152, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0xfe8 00010000:00010000:0.0:1769074788.172627:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.172630:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.172639:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.172663:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 16 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.172677:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97400, rc 0) 00000100:00100000:0.0:1769074788.172742:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:63053:x1855007616939648:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 198us (402us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.172750:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14351 00000100:00100000:2.0:1769074788.175426:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.175512:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939776 00000100:00100000:0.0:1769074788.175524:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939776 00000100:00100000:0.0:1769074788.175584:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14352 00000100:00100000:0.0:1769074788.175594:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:63053:x1855007616939776:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074788.175612:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074788.175634:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c258 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074788.175658:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bd:0x0] initial LVB size: 3145728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x1000 00010000:00010000:0.0:1769074788.175682:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.175685:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.175694:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.175711:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 17 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.175724:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d96e00, rc 0) 00000100:00100000:0.0:1769074788.175792:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:63053:x1855007616939776:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 200us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.175801:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14352 00000100:00100000:1.0:1769074788.181368:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.185035:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616939904 00000100:00100000:2.0:1769074788.185061:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616939904 00000100:00100000:2.0:1769074788.185260:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14353 00000100:00100000:2.0:1769074788.185272:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:63053:x1855007616939904:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.185291:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.185323:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c25f expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074788.185348:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c0:0x0] initial LVB size: 3145728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x1000 00010000:00010000:2.0:1769074788.185372:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.185374:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.185383:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.185401:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 21 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.185414:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718000, rc 0) 00000100:00100000:2.0:1769074788.185491:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:63053:x1855007616939904:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 221us (4133us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.185499:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14353 00000100:00100000:3.0:1769074788.187607:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.197575:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940032 00000100:00100000:0.0:1769074788.197593:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940032 00000100:00100000:0.0:1769074788.197669:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14354 00000100:00100000:0.0:1769074788.197679:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:63053:x1855007616940032:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074788.197698:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074788.197722:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c266 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074788.197746:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2be:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:0.0:1769074788.197769:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.197771:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.197781:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.197798:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 18 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.197826:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d97800, rc 0) 00000100:00100000:0.0:1769074788.198776:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:63053:x1855007616940032:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1093us (11174us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.198785:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14354 00000100:00100000:3.0:1769074788.204255:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.205192:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940160 00000100:00100000:1.0:1769074788.205216:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940160 00000100:00100000:1.0:1769074788.205286:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14355 00000100:00100000:1.0:1769074788.205297:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:63053:x1855007616940160:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074788.205314:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074788.205335:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c26d expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074788.205359:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c1:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:1.0:1769074788.205381:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074788.205383:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074788.205393:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.205741:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.205757:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b053000, rc 0) 00000100:00100000:1.0:1769074788.205953:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:63053:x1855007616940160:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 657us (1708us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074788.205962:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14355 00000100:00100000:3.0:1769074788.212346:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.212462:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940288 00000100:00100000:2.0:1769074788.212473:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940288 00000100:00100000:2.0:1769074788.212544:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14356 00000100:00100000:2.0:1769074788.212554:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:63053:x1855007616940288:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.212572:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.212593:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c274 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074788.212621:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bf:0x0] initial LVB size: 4808704, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x4b0 00010000:00010000:2.0:1769074788.212643:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.212645:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.212654:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.212669:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 19 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.212680:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b718600, rc 0) 00000100:00100000:2.0:1769074788.212738:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:63053:x1855007616940288:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 185us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.212745:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14356 00000100:00100000:3.0:1769074788.224139:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.226845:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940416 00000100:00100000:1.0:1769074788.226861:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940416 00000100:00100000:1.0:1769074788.227008:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14357 00000100:00100000:1.0:1769074788.227019:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:63053:x1855007616940416:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074788.227037:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074788.227070:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd54567 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c27b expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074788.227092:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c2:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x0 00010000:00010000:1.0:1769074788.227113:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074788.227116:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074788.227125:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd54567 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c27b expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.227140:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd54567 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c27b expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.227152:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b052a00, rc 0) 00000100:00100000:1.0:1769074788.227415:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:63053:x1855007616940416:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 397us (4007us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074788.227423:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14357 00000100:00100000:3.0:1769074788.240162:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.242038:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.243335:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940544 00000100:00100000:0.0:1769074788.243350:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940544 00002000:00010000:0.0:1769074788.243422:0:8403:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/703 (4805728->18446744073709551615) 00002000:00100000:0.0:1769074788.243428:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9180 with x1855007616940544 ext(4805728->18446744073709551615) 00002000:00010000:0.0:1769074788.243441:0:8403:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5476f9180. 00000100:00100000:0.0:1769074788.243471:0:8403:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940672 00000100:00100000:0.0:1769074788.243474:0:8403:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940672 00002000:00010000:0.0:1769074788.243486:0:8403:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/706 (4194304->18446744073709551615) 00002000:00100000:0.0:1769074788.243489:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8700 with x1855007616940672 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1769074788.243496:0:8403:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8700. 00000100:00100000:0.0:1769074788.243534:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8712 00000100:00100000:0.0:1769074788.243543:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007616940544:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:0.0:1769074788.243856:0:8403:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4808704 -> 4805728 00002000:00010000:0.0:1769074788.243861:0:8403:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.244086:0:8403:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074788.244089:0:8403:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.244092:0:8403:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074788.244147:0:24028:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8713 00000100:00100000:1.0:1769074788.244158:0:24028:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616940672:12345-192.168.204.60@tcp:10:lfs.0 00000100:00100000:0.0:1769074788.244162:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007616940544:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 620us (4012us total) trans 21474841252 rc 0/0 00000100:00100000:0.0:1769074788.244169:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8712 00002000:00010000:0.0:1769074788.244173:0:8403:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/703 (4805728->18446744073709551615) 00002000:00100000:0.0:1769074788.244176:0:8403:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f9180 with x1855007616940544 ext(4805728->18446744073709551615) 00002000:00010000:0.0:1769074788.244181:0:8403:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b5476f9180. 00002000:00010000:1.0:1769074788.244243:0:24028:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074788.244246:0:24028:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:1.0:1769074788.244248:0:24028:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074788.244250:0:24028:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00002000:00010000:1.0:1769074788.244252:0:24028:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074788.244275:0:24028:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616940672:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 121us (2247us total) trans 17179874583 rc 0/0 00000100:00100000:1.0:1769074788.244282:0:24028:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8713 00002000:00010000:1.0:1769074788.244285:0:24028:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/706 (4194304->18446744073709551615) 00002000:00100000:1.0:1769074788.244287:0:24028:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b5476f8700 with x1855007616940672 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1769074788.244291:0:24028:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b5476f8700. 00000100:00100000:3.0:1769074788.248036:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.248424:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616940800 00000100:00100000:0.0:1769074788.248434:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616940800 00000100:00100000:0.0:1769074788.248495:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13369 00000100:00100000:0.0:1769074788.248505:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63053:x1855007616940800:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074788.248529:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074788.248550:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5456e lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c282 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.248650:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54575 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074788.248665:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54575 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074788.248675:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54575 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074788.248708:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:0.0:1769074788.248713:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074788.248730:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54575 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074788.248753:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547876200/0x7fe2323adfd5456e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c282 expref: 227 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.248771:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54575 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c282 expref: 226 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.248779:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:0.0:1769074788.248782:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074788.248786:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547877e00, rc 0) 00000100:00100000:0.0:1769074788.248900:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:63053:x1855007616940800:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 397us (874us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.248908:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13369 00000100:00100000:0.0:1769074788.255276:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.255347:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941056 00000100:00100000:3.0:1769074788.255366:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941056 00000100:00100000:3.0:1769074788.255426:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13370 00000100:00100000:3.0:1769074788.255435:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:63053:x1855007616941056:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074788.255497:0:8855:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074788.255523:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547877e00/0x7fe2323adfd54575 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c282 expref: 226 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.255540:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.255545:0:8855:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074788.255715:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074788.255729:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074788.255738:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000001:00080000:3.0:1769074788.255783:0:8855:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944012 00000004:00010000:3.0:1769074788.255837:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000002e8c05c reply state 00000000bc2251ec transno 8589944013 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074788.255849:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074788.255855:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.255858:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074788.255861:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074788.255967:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:63053:x1855007616941056:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 533us (701us total) trans 8589944013 rc 0/0 00000100:00100000:3.0:1769074788.255974:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13370 00000020:00080000:3.0:1769074788.255978:0:8855:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1769074788.259692:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.259989:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941184 00000100:00100000:2.0:1769074788.260001:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941184 00000100:00100000:2.0:1769074788.260222:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14358 00000100:00100000:2.0:1769074788.260232:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:63053:x1855007616941184:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.260251:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.260277:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c289 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074788.260301:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bd:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1fe8 00010000:00010000:2.0:1769074788.260322:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.260324:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.260333:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.260364:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.260377:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54b719800, rc 0) 00000100:00100000:2.0:1769074788.260652:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:63053:x1855007616941184:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 420us (967us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.260660:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14358 00000100:00100000:2.0:1769074788.263932:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.264075:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941312 00000100:00100000:3.0:1769074788.264086:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941312 00000100:00100000:3.0:1769074788.264144:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14359 00000100:00100000:3.0:1769074788.264154:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:63053:x1855007616941312:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.264171:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.264192:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c290 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074788.264228:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c3:0x0] initial LVB size: 8388608, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x2000 00010000:00010000:3.0:1769074788.264251:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.264253:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.264262:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 26 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.264278:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 26 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.264289:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88600, rc 0) 00000100:00100000:3.0:1769074788.264353:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:63053:x1855007616941312:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 199us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.264368:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14359 00000100:00100000:0.0:1769074788.267628:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.267787:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941440 00000100:00100000:2.0:1769074788.267797:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941440 00000100:00100000:2.0:1769074788.267882:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14360 00000100:00100000:2.0:1769074788.267892:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:63053:x1855007616941440:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.267909:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.267935:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c297 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074788.267957:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c4:0x0] initial LVB size: 4808704, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x4b0 00010000:00010000:2.0:1769074788.267978:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.267980:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.267988:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 27 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.268003:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 27 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.268021:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:2.0:1769074788.268084:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:63053:x1855007616941440:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 194us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.268092:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14360 00000100:00100000:0.0:1769074788.271793:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.272039:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941568 00000100:00100000:3.0:1769074788.272059:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941568 00000100:00100000:3.0:1769074788.272121:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14361 00000100:00100000:3.0:1769074788.272130:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:63053:x1855007616941568:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.272148:0:27844:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.272171:0:27844:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54598 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c29e expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074788.272197:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c0:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x0 00010000:00010000:3.0:1769074788.272231:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.272234:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.272246:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54598 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c29e expref: 21 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.272262:0:27844:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54598 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c29e expref: 21 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.272275:0:27844:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:3.0:1769074788.272338:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:63053:x1855007616941568:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 209us (557us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.272346:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14361 00000100:00100000:0.0:1769074788.275852:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.276136:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941696 00000100:00100000:2.0:1769074788.276147:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941696 00002000:00010000:2.0:1769074788.276188:0:16436:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/708 (4805728->18446744073709551615) 00002000:00100000:2.0:1769074788.276193:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b424700 with x1855007616941696 ext(4805728->18446744073709551615) 00002000:00010000:2.0:1769074788.276217:0:16436:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b424700. 00000100:00100000:2.0:1769074788.276250:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8714 00000100:00100000:2.0:1769074788.276260:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2409:x1855007616941696:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:2.0:1769074788.276838:0:16436:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4808704 -> 4805728 00002000:00010000:2.0:1769074788.276842:0:16436:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:2.0:1769074788.276845:0:16436:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074788.276847:0:16436:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:2.0:1769074788.276850:0:16436:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:2.0:1769074788.276921:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616941696:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 662us (1078us total) trans 17179874584 rc 0/0 00000100:00100000:2.0:1769074788.276929:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8714 00002000:00010000:2.0:1769074788.276933:0:16436:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/708 (4805728->18446744073709551615) 00002000:00100000:2.0:1769074788.276936:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b424700 with x1855007616941696 ext(4805728->18446744073709551615) 00002000:00010000:2.0:1769074788.276941:0:16436:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b424700. 00000100:00100000:0.0:1769074788.280284:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.280404:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616941824 00000100:00100000:2.0:1769074788.280415:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616941824 00002000:00010000:2.0:1769074788.280455:0:16436:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/704 (4194304->18446744073709551615) 00002000:00100000:2.0:1769074788.280460:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425f80 with x1855007616941824 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1769074788.280472:0:16436:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425f80. 00000100:00100000:2.0:1769074788.280508:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8715 00000100:00100000:2.0:1769074788.280517:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007616941824:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:2.0:1769074788.280671:0:16436:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1769074788.280675:0:16436:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074779 -> 1769074787 00002000:00010000:2.0:1769074788.280677:0:16436:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074788.280680:0:16436:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074779 -> 1769074787 00002000:00010000:2.0:1769074788.280682:0:16436:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074788.280744:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616941824:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 228us (469us total) trans 21474841253 rc 0/0 00000100:00100000:2.0:1769074788.280752:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8715 00002000:00010000:2.0:1769074788.280756:0:16436:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/704 (4194304->18446744073709551615) 00002000:00100000:2.0:1769074788.280759:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b425f80 with x1855007616941824 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1769074788.280764:0:16436:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54b425f80. 00000100:00100000:0.0:1769074788.284511:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.284671:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942080 00000100:00100000:3.0:1769074788.284682:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942080 00000100:00100000:3.0:1769074788.284737:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13371 00000100:00100000:3.0:1769074788.284748:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:63053:x1855007616942080:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074788.284907:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.284931:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.284943:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00010000:3.0:1769074788.285061:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000040d3d727 reply state 0000000065261684 transno 8589944014 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.285075:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.285083:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.285086:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074788.285091:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074788.285193:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:63053:x1855007616942080:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 447us (693us total) trans 8589944014 rc 0/0 00000100:00100000:3.0:1769074788.285211:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13371 00000100:00100000:0.0:1769074788.289878:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.290170:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942208 00000100:00100000:2.0:1769074788.290182:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942208 00000100:00100000:2.0:1769074788.290257:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14362 00000100:00100000:2.0:1769074788.290268:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:63053:x1855007616942208:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.290287:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.290305:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2a5 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.290334:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 29 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074788.290347:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.290349:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.290364:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 29 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.290405:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 29 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074788.290417:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 29 pid: 28519 timeout: 1876 lvb_type: 1 00000100:00100000:2.0:1769074788.290431:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625186560:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074788.290518:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074788.295310:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.295656:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942336 00000100:00100000:3.0:1769074788.295666:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942336 00010000:00010000:3.0:1769074788.295707:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 30 pid: 28519 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.295754:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7155 00000100:00100000:3.0:1769074788.295763:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616942336:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074788.295778:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074788.295798:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 30 pid: 28519 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.295857:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 30 pid: 28519 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.295882:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.295884:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.295892:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.295901:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.295938:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.295952:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625186688:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074788.296039:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00010000:00010000:2.0:1769074788.297138:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd5449c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c227 expref: 30 pid: 28519 timeout: 1876 lvb_type: 1 00000100:00100000:2.0:1769074788.297157:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625186560:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074788.297183:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 29 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.297193:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866800, rc 0) 00000100:00100000:2.0:1769074788.297266:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:63053:x1855007616942208:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6999us (7405us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.297274:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14362 00000100:00100000:1.0:1769074788.305434:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.305667:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942464 00000100:00100000:2.0:1769074788.305677:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942464 00000100:00100000:2.0:1769074788.305737:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14363 00000100:00100000:2.0:1769074788.305747:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:63053:x1855007616942464:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.305763:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.305781:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2ac expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.305846:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074788.305854:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.305857:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.305861:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.305900:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074788.305913:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 23 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:2.0:1769074788.305926:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625186816:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074788.305998:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.311648:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625186688:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074788.311682:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074788.312416:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616942336:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16648us (17110us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.312428:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7155 00000100:00100000:0.0:1769074788.313578:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.313716:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942592 00000100:00100000:3.0:1769074788.313724:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942592 00010000:00010000:3.0:1769074788.313759:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 24 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.313802:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7156 00000100:00100000:3.0:1769074788.313830:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616942592:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074788.313844:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074788.313876:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 24 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.313888:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 24 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.313912:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.313914:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.313920:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.313929:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.313961:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 24 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.313973:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625186944:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074788.314034:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00010000:00010000:2.0:1769074788.317768:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b052000/0x7fe2323adfd544b1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c22e expref: 24 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:2.0:1769074788.317794:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625186816:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074788.317875:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.317891:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866c00, rc 0) 00000100:00100000:2.0:1769074788.318137:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:63053:x1855007616942464:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 12390us (12712us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.318147:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14363 00000100:00100000:3.0:1769074788.321472:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625186944:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074788.321507:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074788.321565:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2409:x1855007616942592:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7734us (7990us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.321575:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7156 00000100:00100000:2.0:1769074788.321896:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.322625:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942720 00000100:00100000:3.0:1769074788.322634:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942720 00000100:00100000:3.0:1769074788.322678:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14364 00000100:00100000:3.0:1769074788.322687:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:63053:x1855007616942720:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.322705:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.322725:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2b3 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.322753:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.322762:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.322764:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.322768:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.322803:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.322854:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.322941:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625187072:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.323019:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.328337:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625187072:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.328628:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.328649:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89600, rc 0) 00000100:00100000:3.0:1769074788.328713:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:63053:x1855007616942720:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6026us (6817us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.328723:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14364 00000100:00100000:1.0:1769074788.331252:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.331473:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942848 00000100:00100000:3.0:1769074788.331483:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942848 00010000:00010000:3.0:1769074788.331612:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.331720:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7157 00000100:00100000:3.0:1769074788.331731:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616942848:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074788.331751:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074788.331773:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.331789:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.331874:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67b053e00/0x7fe2323adfd544a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c235 expref: 23 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.331893:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.331897:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.331905:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.331916:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.331956:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.331971:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625187200:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074788.332082:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.339937:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625187200:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074788.339971:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074788.340031:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007616942848:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8301us (8789us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.340043:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7157 00000100:00100000:2.0:1769074788.341906:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.342012:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616942976 00000100:00100000:3.0:1769074788.342022:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616942976 00000100:00100000:3.0:1769074788.342082:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14365 00000100:00100000:3.0:1769074788.342092:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:63053:x1855007616942976:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.342110:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.342129:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2ba expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.342160:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.342174:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.342177:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.342181:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 29 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.342226:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.342239:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.342252:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625187328:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.342336:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.347590:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625187328:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.347647:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 29 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.347671:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:3.0:1769074788.347740:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:63053:x1855007616942976:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 5649us (5841us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.347750:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14365 00000100:00100000:2.0:1769074788.350993:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.351160:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943104 00000100:00100000:3.0:1769074788.351171:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943104 00010000:00010000:3.0:1769074788.351230:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.351289:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7158 00000100:00100000:3.0:1769074788.351299:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616943104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074788.351317:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074788.351341:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.351355:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.351380:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053c00/0x7fe2323adfd544bf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c23c expref: 29 pid: 27844 timeout: 1876 lvb_type: 1 00010000:00010000:3.0:1769074788.351409:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.351411:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.351419:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 28 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.351428:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 28 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.351477:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 28 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.351491:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625187456:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074788.351589:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b2609c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.358090:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625187456:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074788.358127:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074788.358207:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616943104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 6899us (7217us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.358220:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7158 00000100:00100000:1.0:1769074788.362981:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.363086:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943232 00000100:00100000:3.0:1769074788.363098:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943232 00000100:00100000:3.0:1769074788.363166:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14366 00000100:00100000:3.0:1769074788.363177:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:63053:x1855007616943232:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.363209:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.363229:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2c1 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.363264:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 29 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.363273:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.363276:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.363281:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 29 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.363316:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 29 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.363330:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 29 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.363344:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625187584:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.363763:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:2.0:1769074788.368504:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.368694:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943360 00000100:00100000:0.0:1769074788.368704:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943360 00010000:00010000:0.0:1769074788.368747:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 30 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:0.0:1769074788.368790:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7159 00000100:00100000:0.0:1769074788.368799:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616943360:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074788.368854:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074788.368887:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 30 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:0.0:1769074788.368901:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 30 pid: 9048 timeout: 1876 lvb_type: 1 00010000:00010000:0.0:1769074788.368929:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.368931:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.368939:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.368949:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.368990:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.369003:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625187712:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074788.369086:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00010000:00010000:3.0:1769074788.369465:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052e00/0x7fe2323adfd544aa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c243 expref: 30 pid: 9048 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.369479:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625187584:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.369508:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 29 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.369517:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89c00, rc 0) 00000100:00100000:3.0:1769074788.369555:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:63053:x1855007616943232:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6381us (6586us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.369563:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14366 00000100:00100000:2.0:1769074788.374668:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.374893:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943488 00000100:00100000:3.0:1769074788.374906:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943488 00000100:00100000:3.0:1769074788.374973:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14367 00000100:00100000:3.0:1769074788.374983:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:63053:x1855007616943488:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.375003:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.375022:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2c8 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.375063:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.375071:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.375073:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.375078:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.375116:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.375129:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 1876 lvb_type: 1 00000100:00100000:3.0:1769074788.375143:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625187840:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.375234:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260c00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074788.376484:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625187712:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074788.376506:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074788.376577:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616943360:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7777us (8081us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.376587:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7159 00000100:00100000:3.0:1769074788.381851:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625187840:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.381909:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->8388607] (req 2097152->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 23 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.381930:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:3.0:1769074788.382007:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:63053:x1855007616943488:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 7023us (7348us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.382016:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14367 00000100:00100000:2.0:1769074788.386162:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.386986:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943616 00000100:00100000:0.0:1769074788.386998:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943616 00010000:00010000:0.0:1769074788.387041:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 1876 lvb_type: 1 00000100:00100000:0.0:1769074788.387149:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7160 00000100:00100000:0.0:1769074788.387160:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616943616:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074788.387179:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074788.387215:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 1876 lvb_type: 1 00010000:00010000:0.0:1769074788.387231:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 1876 lvb_type: 1 00010000:00010000:0.0:1769074788.387260:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602b800/0x7fe2323adfd544b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c24a expref: 23 pid: 8396 timeout: 1876 lvb_type: 1 00010000:00010000:0.0:1769074788.387275:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.387277:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.387284:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.387294:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.387330:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 22 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.387344:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625187968:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074788.387454:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074788.392585:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.392611:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.392642:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943872 00000100:00100000:0.0:1769074788.392651:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943872 00000100:00100000:0.0:1769074788.392694:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944128 00000100:00100000:0.0:1769074788.392698:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944128 00000100:00100000:0.0:1769074788.392719:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14368 00000100:00100000:0.0:1769074788.392729:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616943872:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074788.392904:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074788.392908:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.392911:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.392913:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074776 -> 1769074787 00002000:00010000:0.0:1769074788.392916:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074788.392972:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616943872:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 243us (397us total) trans 21474841254 rc 0/0 00000100:00100000:0.0:1769074788.392980:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14368 00000100:00100000:0.0:1769074788.392998:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14369 00000100:00100000:0.0:1769074788.393004:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616944128:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074788.393086:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074788.393090:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.393092:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.393094:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074776 -> 1769074787 00002000:00010000:0.0:1769074788.393097:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:0.0:1769074788.393120:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616944128:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 116us (508us total) trans 17179874585 rc 0/0 00000100:00100000:0.0:1769074788.393126:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14369 00000100:00100000:0.0:1769074788.393166:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.393187:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944384 00000100:00100000:0.0:1769074788.393191:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944384 00000100:00100000:0.0:1769074788.393235:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14370 00000100:00100000:0.0:1769074788.393241:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2408:x1855007616944384:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074788.393294:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074788.393297:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074777 -> 1769074780 00002000:00010000:0.0:1769074788.393299:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.393302:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074777 -> 1769074787 00002000:00010000:0.0:1769074788.393304:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:0.0:1769074788.393326:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616944384:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 85us (160us total) trans 21474841255 rc 0/0 00000100:00100000:0.0:1769074788.393333:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14370 00000100:00100000:0.0:1769074788.393394:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.393409:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616943744 00000100:00100000:0.0:1769074788.393413:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616943744 00000100:00100000:0.0:1769074788.393432:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14371 00000100:00100000:0.0:1769074788.393438:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616943744:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074788.393493:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:0.0:1769074788.393496:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074772 -> 1769074780 00002000:00010000:0.0:1769074788.393498:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime from disk: 1769074770 -> 1769074780 00002000:00010000:0.0:1769074788.393500:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074772 -> 1769074787 00002000:00010000:0.0:1769074788.393502:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb blocks from disk: 2024 -> 2024 00000100:00100000:0.0:1769074788.393523:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616943744:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 85us (129us total) trans 17179874586 rc 0/0 00000100:00100000:0.0:1769074788.393530:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14371 00000100:00100000:0.0:1769074788.393571:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.393586:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944000 00000100:00100000:0.0:1769074788.393591:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944000 00000100:00100000:0.0:1769074788.393610:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14372 00000100:00100000:0.0:1769074788.393616:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616944000:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074788.393664:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074788.393667:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.393669:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074776 -> 1769074780 00002000:00010000:0.0:1769074788.393672:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074775 -> 1769074787 00002000:00010000:0.0:1769074788.393675:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074788.393696:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616944000:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 79us (124us total) trans 21474841256 rc 0/0 00000100:00100000:0.0:1769074788.393703:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14372 00000100:00100000:0.0:1769074788.393753:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.393768:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944256 00000100:00100000:0.0:1769074788.393772:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944256 00000100:00100000:0.0:1769074788.393792:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14373 00000100:00100000:0.0:1769074788.393798:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616944256:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074788.393883:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074788.393887:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:0.0:1769074788.393889:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime from disk: 1769074777 -> 1769074780 00002000:00010000:0.0:1769074788.393891:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074780 -> 1769074787 00002000:00010000:0.0:1769074788.393893:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb blocks from disk: 5296 -> 5296 00000100:00100000:0.0:1769074788.393914:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616944256:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 116us (161us total) trans 17179874587 rc 0/0 00000100:00100000:0.0:1769074788.393921:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14373 00000100:00100000:1.0:1769074788.396319:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6498:1855007625187968:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074788.396348:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074788.396424:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616943616:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 9264us (10271us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074788.396436:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7160 00000100:00100000:0.0:1769074788.401267:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.401510:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944640 00000100:00100000:3.0:1769074788.401532:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944640 00000100:00100000:3.0:1769074788.401595:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4696 00000100:00100000:3.0:1769074788.401605:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:63053:x1855007616944640:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:3.0:1769074788.401676:0:6516:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x200000405:0x1fb:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54448 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c212 expref: 230 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.401731:0:6516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6516 initiator: MDT0 00010000:00010000:3.0:1769074788.401747:0:6516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 230 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.401757:0:6516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT0 00010000:00010000:3.0:1769074788.401795:0:6516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 230 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.401835:0:6516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 230 pid: 8855 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074788.401850:0:6516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:6516:1855007625188096:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.401943:0:6516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.406951:0:6516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:6516:1855007625188096:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.406995:0:6516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT0 00000100:00100000:0.0:1769074788.408351:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.408453:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944768 00000100:00100000:1.0:1769074788.408463:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944768 00010000:00010000:1.0:1769074788.408504:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 231 pid: 8855 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074788.408553:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7161 00000100:00100000:1.0:1769074788.408563:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+231:2409:x1855007616944768:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074788.408579:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074788.408585:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 231 pid: 8855 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074788.408597:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 231 pid: 8855 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074788.408623:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd54487 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c219 expref: 231 pid: 8855 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074788.408642:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:1.0:1769074788.408647:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT0 00010000:00010000:1.0:1769074788.408657:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6516 initiator: MDT0 00010000:00010000:1.0:1769074788.408665:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6516 initiator: MDT0 00010000:00010000:1.0:1769074788.408697:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074788.408754:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:2409:x1855007616944768:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 192us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074788.408762:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7161 00010000:00010000:3.0:1769074788.408894:0:6516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT0 00010000:00010000:3.0:1769074788.408903:0:6516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT0 00000040:00080000:3.0:1769074788.409367:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2447, 88 off209552 00000004:00080000:3.0:1769074788.409377:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251569792ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409422:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2443, 88 off208936 00000004:00080000:3.0:1769074788.409428:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251621237ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409455:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2444, 88 off209024 00000004:00080000:3.0:1769074788.409461:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251653917ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409472:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2448, 88 off209640 00000004:00080000:3.0:1769074788.409477:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251669672ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409486:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2449, 88 off209728 00000004:00080000:3.0:1769074788.409490:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251683014ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409501:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2445, 88 off209112 00000004:00080000:3.0:1769074788.409504:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251697547ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409514:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2446, 88 off209200 00000004:00080000:3.0:1769074788.409518:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251711282ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409528:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2447, 88 off209288 00000004:00080000:3.0:1769074788.409531:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251724185ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409541:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2450, 88 off209816 00000004:00080000:3.0:1769074788.409545:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251738358ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409556:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2448, 88 off209376 00000004:00080000:3.0:1769074788.409560:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251752810ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409569:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2451, 88 off209904 00000004:00080000:3.0:1769074788.409573:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251766297ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409583:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2449, 88 off209464 00000004:00080000:3.0:1769074788.409587:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251780162ns, next at 1706953247299ns, rc = 0 00000040:00080000:3.0:1769074788.409597:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2452, 88 off209992 00000004:00080000:3.0:1769074788.409601:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251793854ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409610:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2453, 88 off210080 00000004:00080000:3.0:1769074788.409614:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251807107ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409623:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2454, 88 off210168 00000004:00080000:3.0:1769074788.409627:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251819702ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409635:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2455, 88 off210256 00000004:00080000:3.0:1769074788.409639:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1776251831984ns, next at 1706951295843ns, rc = 0 00000040:00080000:3.0:1769074788.409649:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2450, 88 off209552 00000004:00080000:3.0:1769074788.409653:0:6516:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1776251845918ns, next at 1706953247299ns, rc = 0 00000001:00080000:3.0:1769074788.413853:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944015 is committed 00000004:00080000:3.0:1769074788.413875:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1954 committed 00000004:00080000:3.0:1769074788.413885:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1958 committed 00000004:00080000:3.0:1769074788.413893:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1957 committed 00000004:00080000:3.0:1769074788.413895:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1956 committed 00000004:00080000:3.0:1769074788.413897:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1955 committed 00000004:00080000:3.0:1769074788.413899:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1953 committed 00000004:00080000:3.0:1769074788.413901:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1954 committed 00000004:00080000:3.0:1769074788.413903:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1952 committed 00000004:00080000:3.0:1769074788.413905:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1953 committed 00000004:00080000:3.0:1769074788.413907:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1951 committed 00000004:00080000:3.0:1769074788.413909:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1950 committed 00000004:00080000:3.0:1769074788.413911:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1949 committed 00000004:00080000:3.0:1769074788.413913:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1952 committed 00000004:00080000:3.0:1769074788.413914:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1951 committed 00000004:00080000:3.0:1769074788.413916:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1948 committed 00000004:00080000:3.0:1769074788.413918:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1947 committed 00000004:00080000:3.0:1769074788.413919:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1950 committed 00000004:00010000:3.0:1769074788.413976:0:6516:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000001d27987d reply state 0000000004a374e6 transno 8589944015 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT0 00010000:00010000:3.0:1769074788.414002:0:6516:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT0 00010000:00010000:3.0:1769074788.414011:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.414015:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:3.0:1769074788.414019:0:6516:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1769074788.414027:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.414029:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074788.414038:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54448 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c212 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00080000:3.0:1769074788.414162:0:6516:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1769074788.414175:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63053:x1855007616944640:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 12571us (12917us total) trans 8589944015 rc 0/0 00000100:00100000:3.0:1769074788.414184:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4696 00000100:00100000:1.0:1769074788.414422:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625188480:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074788.414468:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074788.414498:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625188992:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074788.414515:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074788.414548:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1769074788.414555:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625188608:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074788.414574:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074788.414591:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625189120:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074788.414609:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074788.414641:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188480 00000100:00100000:1.0:1769074788.414648:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188480 00000100:00100000:1.0:1769074788.414686:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188992 00000100:00100000:1.0:1769074788.414689:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188992 00000100:00100000:1.0:1769074788.414703:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188608 00000100:00100000:1.0:1769074788.414706:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188608 00000100:00100000:1.0:1769074788.414721:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189120 00000100:00100000:1.0:1769074788.414724:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189120 00000100:00100000:1.0:1769074788.414745:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14374 00000100:00100000:1.0:1769074788.414752:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625188480:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074788.414909:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074788.414913:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:1.0:1769074788.414916:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb atime from disk: 1769074780 -> 1769074780 00002000:00010000:1.0:1769074788.414918:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:1.0:1769074788.414920:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bc:0x0] updating lvb blocks from disk: 4104 -> 4104 00000100:00100000:1.0:1769074788.414952:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625188480:12345-0@lo:2:osp-syn-0-0.0 Request processed in 200us (484us total) trans 21474841257 rc 0/0 00000100:00100000:1.0:1769074788.414959:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14374 00000100:00100000:1.0:1769074788.414974:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14375 00000100:00100000:1.0:1769074788.414980:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625188992:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074788.415028:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:1.0:1769074788.415031:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:1.0:1769074788.415034:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074788.415036:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:1.0:1769074788.415039:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bf:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:1.0:1769074788.415074:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625188992:12345-0@lo:2:osp-syn-0-0.0 Request processed in 94us (557us total) trans 21474841258 rc 0/0 00000100:00100000:1.0:1769074788.415080:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14375 00000100:00100000:1.0:1769074788.415093:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14376 00000100:00100000:1.0:1769074788.415098:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625188608:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074788.415143:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074788.415147:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:1.0:1769074788.415149:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074788.415152:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:1.0:1769074788.415154:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2b9:0x0] updating lvb blocks from disk: 4072 -> 4072 00000100:00100000:1.0:1769074788.415177:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625188608:12345-0@lo:2:osp-syn-0-0.0 Request processed in 79us (602us total) trans 21474841259 rc 0/0 00000100:00100000:1.0:1769074788.415183:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14376 00000100:00100000:1.0:1769074788.415196:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14377 00000100:00100000:1.0:1769074788.415201:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625189120:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074788.415244:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1769074788.415247:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:1.0:1769074788.415250:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074788.415252:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:1.0:1769074788.415254:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c0:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074788.415277:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3651:x1855007625189120:12345-0@lo:2:osp-syn-0-0.0 Request processed in 77us (668us total) trans 21474841260 rc 0/0 00000100:00100000:1.0:1769074788.415283:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14377 00000100:00100000:1.0:1769074788.415332:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841247 gen 2 00000004:00080000:1.0:1769074788.415337:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2aa00, transno 21474841240 00000004:00080000:1.0:1769074788.415346:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2b100, transno 21474841241 00000004:00080000:1.0:1769074788.415349:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f28380, transno 21474841242 00000004:00080000:1.0:1769074788.415351:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f28000, transno 21474841243 00000004:00080000:1.0:1769074788.415354:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f29180, transno 21474841244 00000004:00080000:1.0:1769074788.415356:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2bb80, transno 21474841245 00000004:00080000:1.0:1769074788.415359:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2a300, transno 21474841246 00000004:00080000:1.0:1769074788.415361:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2ad80, transno 21474841247 00000100:00100000:1.0:1769074788.415367:0:3651:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b54b425180 x1855007625188608/t21474841259(21474841259) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074804 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1769074788.415378:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54b425180/2, rc 0, transno 21474841259 00000100:00100000:1.0:1769074788.415382:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625188608:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074788.415393:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475df800/2, rc 0, transno 21474841260 00000100:00100000:1.0:1769074788.415396:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625189120:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074788.415403:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef5500/2, rc 0, transno 21474841257 00000100:00100000:1.0:1769074788.415406:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625188480:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074788.415413:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475ddc00/2, rc 0, transno 21474841258 00000100:00100000:1.0:1769074788.415416:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625188992:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1769074788.415523:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 8 in progress, 4 in flight, 8 done 00000100:00100000:0.0:1769074788.416023:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188352:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074788.416073:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074788.416106:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188864:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074788.416128:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074788.416144:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1769074788.416150:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188224:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074788.416171:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074788.416186:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188736:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074788.416202:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074788.416224:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188352 00000100:00100000:0.0:1769074788.416230:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188352 00000100:00100000:0.0:1769074788.416259:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188864 00000100:00100000:0.0:1769074788.416261:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188864 00000100:00100000:0.0:1769074788.416274:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188224 00000100:00100000:0.0:1769074788.416276:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188224 00000100:00100000:0.0:1769074788.416286:0:10284:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625188736 00000100:00100000:0.0:1769074788.416288:0:10284:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625188736 00000100:00100000:0.0:1769074788.416302:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14378 00000100:00100000:0.0:1769074788.416309:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625188352:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074788.416391:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074788.416395:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:0.0:1769074788.416397:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb atime from disk: 1769074780 -> 1769074780 00002000:00010000:0.0:1769074788.416399:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.416401:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074788.416434:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625188352:12345-0@lo:2:osp-syn-0-0.0 Request processed in 126us (361us total) trans 21474841261 rc 0/0 00000100:00100000:0.0:1769074788.416453:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14378 00000100:00100000:0.0:1769074788.416468:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14379 00000100:00100000:0.0:1769074788.416474:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625188864:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074788.416532:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074788.416535:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.416537:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074788.416540:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.416542:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2be:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074788.416566:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625188864:12345-0@lo:2:osp-syn-0-0.0 Request processed in 93us (437us total) trans 21474841262 rc 0/0 00000100:00100000:0.0:1769074788.416573:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14379 00000100:00100000:0.0:1769074788.416586:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14380 00000100:00100000:0.0:1769074788.416591:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625188224:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074788.416637:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074788.416640:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:0.0:1769074788.416643:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb atime from disk: 1769074780 -> 1769074780 00002000:00010000:0.0:1769074788.416644:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.416647:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2ba:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074788.416671:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625188224:12345-0@lo:2:osp-syn-0-0.0 Request processed in 80us (499us total) trans 21474841263 rc 0/0 00000100:00100000:0.0:1769074788.416677:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14380 00000100:00100000:0.0:1769074788.416689:0:10284:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14381 00000100:00100000:0.0:1769074788.416695:0:10284:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1855007625188736:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1769074788.416737:0:10284:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:0.0:1769074788.416740:0:10284:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.416742:0:10284:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074788.416744:0:10284:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.416746:0:10284:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2bd:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074788.416769:0:10284:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625188736:12345-0@lo:2:osp-syn-0-0.0 Request processed in 75us (565us total) trans 21474841264 rc 0/0 00000100:00100000:0.0:1769074788.416775:0:10284:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14381 00000004:00080000:0.0:1769074788.416846:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef6a00/2, rc 0, transno 21474841263 00000100:00100000:0.0:1769074788.416851:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188224:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074788.416872:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dd880/2, rc 0, transno 21474841264 00000100:00100000:0.0:1769074788.416876:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188736:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074788.416884:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef6680/2, rc 0, transno 21474841261 00000100:00100000:0.0:1769074788.416888:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188352:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1769074788.416894:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5475dfb80/2, rc 0, transno 21474841262 00000100:00100000:0.0:1769074788.416897:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625188864:0@lo:2:osp-syn-0-0.0 00010000:00010000:2.0:1769074788.418349:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418365:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418373:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418378:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.418383:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418402:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418419:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074788.418423:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074788.418428:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418434:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54536 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418465:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418471:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418475:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418480:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.418482:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418487:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418493:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074788.418496:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074788.418498:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418503:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5457c lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.418514:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418530:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418536:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418540:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.418543:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418547:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418554:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074788.418556:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074788.418558:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418563:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5459f lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074788.418582:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT0 00010000:00010000:2.0:1769074788.418587:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6516 initiator: MDT0 00010000:00010000:2.0:1769074788.418592:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT0 00010000:00010000:2.0:1769074788.418596:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547d96000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.418598:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT0 00010000:00010000:2.0:1769074788.418604:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6516 initiator: MDT0 00010000:00010000:2.0:1769074788.418615:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074788.418617:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074788.418619:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT0 00010000:00010000:2.0:1769074788.418624:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545d0 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT0 00000100:00100000:2.0:1769074788.418772:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625189632:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.418853:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074788.419149:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625189248:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.419327:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625190144:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.419352:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074788.419375:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1769074788.419381:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d100000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625189504:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.419403:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074788.419419:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625190016:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.419800:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074788.420068:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625189376:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074788.420103:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074788.424148:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189632 00000100:00100000:3.0:1769074788.424165:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189632 00000100:00100000:3.0:1769074788.424227:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625190144 00000100:00100000:3.0:1769074788.424231:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625190144 00000100:00100000:3.0:1769074788.424248:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189504 00000100:00100000:3.0:1769074788.424251:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189504 00000100:00100000:3.0:1769074788.424260:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625190016 00000100:00100000:3.0:1769074788.424262:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625190016 00000100:00100000:3.0:1769074788.424270:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189376 00000100:00100000:3.0:1769074788.424272:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189376 00000100:00100000:3.0:1769074788.424288:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14382 00000100:00100000:3.0:1769074788.424296:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625189632:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074788.424466:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb size from disk: 3145728 -> 3145728 00002000:00010000:3.0:1769074788.424469:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.424472:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769074788.424474:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.424477:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c0:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:3.0:1769074788.424555:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625189632:12345-0@lo:2:osp-syn-1-0.0 Request processed in 260us (5701us total) trans 17179874588 rc 0/0 00000100:00100000:3.0:1769074788.424563:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14382 00000100:00100000:3.0:1769074788.424580:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14383 00000100:00100000:3.0:1769074788.424587:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625190144:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074788.424644:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb size from disk: 8388608 -> 8388608 00002000:00010000:3.0:1769074788.424648:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.424650:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769074788.424652:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.424654:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c3:0x0] updating lvb blocks from disk: 8192 -> 8192 00000100:00100000:3.0:1769074788.424680:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625190144:12345-0@lo:2:osp-syn-1-0.0 Request processed in 94us (5328us total) trans 17179874589 rc 0/0 00000100:00100000:3.0:1769074788.424687:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14383 00000100:00100000:3.0:1769074788.424699:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14384 00000100:00100000:3.0:1769074788.424706:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625189504:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074788.424754:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:3.0:1769074788.424758:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:3.0:1769074788.424760:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb atime from disk: 1769074780 -> 1769074780 00002000:00010000:3.0:1769074788.424762:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.424764:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bf:0x0] updating lvb blocks from disk: 5296 -> 5296 00000100:00100000:3.0:1769074788.424789:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625189504:12345-0@lo:2:osp-syn-1-0.0 Request processed in 84us (5385us total) trans 17179874590 rc 0/0 00000100:00100000:3.0:1769074788.424796:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14384 00000100:00100000:3.0:1769074788.424838:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14385 00000100:00100000:3.0:1769074788.424845:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625190016:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074788.425303:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1769074788.425307:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb mtime from disk: 1769074779 -> 1769074779 00002000:00010000:3.0:1769074788.425309:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769074788.425312:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb ctime from disk: 1769074779 -> 1769074779 00002000:00010000:3.0:1769074788.425314:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bd:0x0] updating lvb blocks from disk: 8168 -> 8168 00000100:00100000:3.0:1769074788.425345:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625190016:12345-0@lo:2:osp-syn-1-0.0 Request processed in 500us (5906us total) trans 17179874591 rc 0/0 00000100:00100000:3.0:1769074788.425352:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14385 00000100:00100000:3.0:1769074788.425365:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14386 00000100:00100000:3.0:1769074788.425371:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3650:x1855007625189376:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074788.425434:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:3.0:1769074788.425437:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:3.0:1769074788.425439:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb atime from disk: 1769074780 -> 1769074780 00002000:00010000:3.0:1769074788.425441:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.425444:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2be:0x0] updating lvb blocks from disk: 2056 -> 2056 00000100:00100000:3.0:1769074788.425470:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+10:3650:x1855007625189376:12345-0@lo:2:osp-syn-1-0.0 Request processed in 99us (5366us total) trans 17179874592 rc 0/0 00000100:00100000:3.0:1769074788.425476:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14386 00000100:00100000:2.0:1769074788.425826:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074788.425977:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189248 00000100:00100000:3.0:1769074788.425984:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189248 00000100:00100000:3.0:1769074788.426017:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14387 00000100:00100000:3.0:1769074788.426024:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+10:3649:x1855007625189248:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769074788.426099:0:28519:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:3.0:1769074788.426103:0:28519:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb mtime from disk: 1769074780 -> 1769074780 00002000:00010000:3.0:1769074788.426105:0:28519:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb atime from disk: 1769074780 -> 1769074780 00002000:00010000:3.0:1769074788.426108:0:28519:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:3.0:1769074788.426110:0:28519:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2bc:0x0] updating lvb blocks from disk: 2024 -> 2024 00000100:00100000:3.0:1769074788.426142:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+11:3649:x1855007625189248:12345-0@lo:2:osp-syn-1-0.0 Request processed in 118us (338us total) trans 17179874593 rc 0/0 00000100:00100000:3.0:1769074788.426149:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14387 00000100:00100000:2.0:1769074788.426190:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625189760:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.426335:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074788.426431:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874577 gen 2 00000004:00080000:2.0:1769074788.426438:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475de300, transno 17179874569 00000004:00080000:2.0:1769074788.426449:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ded80, transno 17179874570 00000004:00080000:2.0:1769074788.426452:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dea00, transno 17179874571 00000004:00080000:2.0:1769074788.426454:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dca80, transno 17179874572 00000004:00080000:2.0:1769074788.426457:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df480, transno 17179874573 00000004:00080000:2.0:1769074788.426459:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc000, transno 17179874574 00000004:00080000:2.0:1769074788.426462:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dc700, transno 17179874575 00000004:00080000:2.0:1769074788.426465:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddf80, transno 17179874576 00000004:00080000:2.0:1769074788.426467:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b542faad80, transno 17179874577 00000100:00100000:2.0:1769074788.426474:0:3649:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b54d100a80 x1855007625189248/t17179874593(17179874593) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074804 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1769074788.426485:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d100a80/2, rc 0, transno 17179874593 00000100:00100000:2.0:1769074788.426490:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625189248:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074788.426783:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 8 in progress, 7 in flight, 9 done 00000100:00100000:2.0:1769074788.426857:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625190272:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.426974:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1769074788.427222:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d100000/2, rc 0, transno 17179874590 00000100:00100000:2.0:1769074788.427227:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625189504:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074788.427236:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d102a00/2, rc 0, transno 17179874591 00000100:00100000:2.0:1769074788.427239:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625190016:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074788.427247:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d101180/2, rc 0, transno 17179874588 00000100:00100000:2.0:1769074788.427250:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625189632:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074788.427258:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d100700/2, rc 0, transno 17179874589 00000100:00100000:2.0:1769074788.427261:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d100700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625190144:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074788.427507:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189760 00000100:00100000:0.0:1769074788.427514:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189760 00000100:00100000:0.0:1769074788.427561:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625190272 00000100:00100000:0.0:1769074788.427565:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625190272 00000100:00100000:0.0:1769074788.427584:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14388 00000100:00100000:0.0:1769074788.427592:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625189760:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074788.427702:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074788.427706:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.427709:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074788.427711:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.427713:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c1:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074788.427756:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625189760:12345-0@lo:2:osp-syn-1-0.0 Request processed in 165us (1420us total) trans 17179874594 rc 0/0 00000100:00100000:0.0:1769074788.427763:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14388 00000100:00100000:0.0:1769074788.427778:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14389 00000100:00100000:0.0:1769074788.427784:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+12:3649:x1855007625190272:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074788.427851:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb size from disk: 4805728 -> 4805728 00002000:00010000:0.0:1769074788.427855:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.427857:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074788.427859:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.427873:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c4:0x0] updating lvb blocks from disk: 1200 -> 1200 00000100:00100000:0.0:1769074788.427898:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625190272:12345-0@lo:2:osp-syn-1-0.0 Request processed in 113us (923us total) trans 17179874595 rc 0/0 00000100:00100000:0.0:1769074788.427904:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14389 00000004:00080000:0.0:1769074788.428033:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d101500/2, rc 0, transno 17179874592 00000100:00100000:0.0:1769074788.428038:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625189376:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074788.428060:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625189888:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074788.428085:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074788.428104:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625189888 00000100:00100000:0.0:1769074788.428108:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625189888 00000100:00100000:0.0:1769074788.428130:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14390 00000100:00100000:0.0:1769074788.428136:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625189888:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074788.428185:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1769074788.428188:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb mtime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.428190:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074788.428193:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb ctime from disk: 1769074787 -> 1769074787 00002000:00010000:0.0:1769074788.428195:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074788.428537:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+14:3650:x1855007625189888:12345-0@lo:2:osp-syn-1-0.0 Request processed in 400us (449us total) trans 17179874596 rc 0/0 00000100:00100000:0.0:1769074788.428544:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14390 00000004:00080000:0.0:1769074788.428574:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d101c00/2, rc 0, transno 17179874596 00000100:00100000:0.0:1769074788.428578:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625189888:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074788.428642:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000004:00080000:2.0:1769074788.429196:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a271880/2, rc 0, transno 17179874595 00000100:00100000:2.0:1769074788.429376:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625190272:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074788.429467:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d103100/2, rc 0, transno 17179874594 00000100:00100000:2.0:1769074788.429471:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625189760:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074788.429490:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616944896 00000100:00100000:2.0:1769074788.429496:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616944896 00000100:00100000:2.0:1769074788.429616:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13372 00000100:00100000:2.0:1769074788.429624:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+220:63053:x1855007616944896:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.429644:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.429662:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd545d7 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2cf expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.429712:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd545d7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2cf expref: 221 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.429726:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd545d7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2cf expref: 221 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.429744:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074788.429747:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074788.429750:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867200, rc 0) 00000100:00100000:2.0:1769074788.429798:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:63053:x1855007616944896:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 175us (1158us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.429825:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13372 00000100:00100000:0.0:1769074788.433011:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.433344:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616945152 00000100:00100000:3.0:1769074788.433355:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616945152 00000100:00100000:3.0:1769074788.433460:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4697 00000100:00100000:3.0:1769074788.433470:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:63053:x1855007616945152:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074788.433715:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63053:x1855007616945152:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 246us (712us total) trans 8589944016 rc 0/0 00000100:00100000:3.0:1769074788.433724:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4697 00000100:00100000:0.0:1769074788.461499:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.461888:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616945408 00000100:00100000:3.0:1769074788.461900:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616945408 00000100:00100000:3.0:1769074788.461964:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13373 00000100:00100000:3.0:1769074788.461975:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63056:x1855007616945408:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.462001:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.462024:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545de lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2d6 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.462112:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462129:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462140:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462155:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462162:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462168:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462252:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462260:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462265:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462271:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462278:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074788.462281:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462289:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462301:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074788.462304:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074788.462309:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462315:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd545e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462328:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462334:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462340:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462345:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074788.462348:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462353:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462360:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b681690700) 00010000:00010000:3.0:1769074788.462362:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074788.462365:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462370:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd545ec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.462402:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545de lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2d6 expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.462411:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074788.462414:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074788.462418:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96000/0x7fe2323adfd545de lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2d6 expref: 223 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.462427:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547d96000, rc 0) 00000100:00100000:3.0:1769074788.462503:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63056:x1855007616945408:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 530us (1016us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074788.462511:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13373 00000100:00100000:1.0:1769074788.469720:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.469853:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616945536 00000100:00100000:2.0:1769074788.469870:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616945536 00000100:00100000:2.0:1769074788.469919:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13374 00000100:00100000:2.0:1769074788.469927:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63056:x1855007616945536:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074788.469994:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470010:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470021:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470034:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470040:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470057:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470128:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470136:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470143:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074788.470185:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00000004:00200000:2.0:1769074788.470188:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074788.470200:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470213:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470219:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470224:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470230:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470239:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.470243:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470250:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470261:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074788.470265:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074788.470269:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470275:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd545f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470289:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470295:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470300:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470304:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.470307:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470313:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470320:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b64effa300) 00010000:00010000:2.0:1769074788.470322:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074788.470326:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470331:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866e00/0x7fe2323adfd545fa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470344:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470349:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470354:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470359:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074788.470362:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470367:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470373:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074788.470376:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074788.470378:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074788.470383:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866000/0x7fe2323adfd54601 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000100:00100000:2.0:1769074788.470465:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63056:x1855007616945536:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 537us (753us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.470472:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13374 00000100:00100000:0.0:1769074788.526114:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.526222:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616945792 00000100:00100000:3.0:1769074788.526235:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616945792 00000100:00100000:3.0:1769074788.526300:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13375 00000100:00100000:3.0:1769074788.526311:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63062:x1855007616945792:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.526339:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.526364:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54608 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2dd expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.526465:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526480:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526491:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526506:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526513:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526519:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526637:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5461d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526646:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5461d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526653:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5461d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074788.526662:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:3.0:1769074788.526740:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526747:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526753:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526760:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526766:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074788.526770:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526777:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526790:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074788.526793:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074788.526797:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526803:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45800/0x7fe2323adfd5460f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526856:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526872:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526878:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526887:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba44000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074788.526890:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526896:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526903:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b681690700) 00010000:00010000:3.0:1769074788.526905:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074788.526908:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526914:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44000/0x7fe2323adfd54616 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074788.526951:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45a00/0x7fe2323adfd54608 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2dd expref: 225 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.526965:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5461d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2dd expref: 224 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.526972:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074788.526974:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074788.526977:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54ba44600, rc 0) 00000100:00100000:3.0:1769074788.527053:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63062:x1855007616945792:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 735us (941us total) trans 8589944017 rc 0/0 00000100:00100000:3.0:1769074788.527061:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13375 00000100:00100000:0.0:1769074788.575949:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.576290:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616945920 00000100:00100000:3.0:1769074788.576300:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616945920 00000100:00100000:3.0:1769074788.576371:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8716 00000100:00100000:3.0:1769074788.576380:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616945920:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074788.576429:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.576451:0:28448:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.576460:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074788.576462:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074788.576465:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.576499:0:28448:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.576513:0:28448:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 31 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074788.576528:0:28448:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_015:lustre-OST0001_UUID:28448:1855007625190400:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.576611:0:28448:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64133a780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074788.581989:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.582050:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.582206:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946048 00000100:00100000:2.0:1769074788.582218:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946048 00000100:00100000:2.0:1769074788.582285:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946176 00000100:00100000:2.0:1769074788.582288:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946176 00000100:00100000:2.0:1769074788.582330:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8717 00000100:00100000:2.0:1769074788.582340:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616946048:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074788.582380:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.582410:0:16436:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.582419:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074788.582421:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074788.582426:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.582463:0:16436:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 25 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.582475:0:16436:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 25 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:2.0:1769074788.582490:0:16436:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273b80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_003:lustre-OST0000_UUID:16436:1855007625190528:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074788.582576:0:16436:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f36c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074788.583069:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8718 00000100:00100000:3.0:1769074788.583081:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616946176:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074788.583106:0:28444:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.583120:0:28444:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 32 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.583128:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074788.583130:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074788.583133:0:28444:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.583159:0:28444:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 32 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.583168:0:28444:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 32 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074788.583179:0:28444:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_012:lustre-OST0001_UUID:28444:1855007625190656:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074788.583238:0:28444:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b653805c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074788.590167:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.590207:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.590246:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.590269:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.590293:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946688 00000100:00100000:1.0:1769074788.590302:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946688 00000100:00100000:1.0:1769074788.590357:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946304 00000100:00100000:1.0:1769074788.590361:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946304 00000100:00100000:1.0:1769074788.590383:0:28445:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946816 00000100:00100000:1.0:1769074788.590398:0:28445:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946816 00000100:00100000:1.0:1769074788.590423:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8719 00000100:00100000:1.0:1769074788.590433:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616946688:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074788.590484:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590503:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 26 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590511:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074788.590513:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074788.590516:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590549:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 26 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590560:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 26 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074788.590574:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625190784:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074788.590652:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4719c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074788.590674:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8720 00000100:00100000:1.0:1769074788.590682:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2409:x1855007616946304:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074788.590700:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590712:0:8404:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 34 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590718:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074788.590720:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074788.590723:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590742:0:8404:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 34 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590751:0:8404:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 34 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074788.590761:0:8404:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0001_UUID:8404:1855007625190912:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074788.590785:0:8404:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b471c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074788.590801:0:8403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8721 00000100:00100000:1.0:1769074788.590843:0:8403:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2407:x1855007616946816:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074788.590859:0:8403:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590868:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074788.590870:0:8403:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074788.590872:0:8403:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.590881:0:8403:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074788.590916:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.590977:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074788.590996:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947072 00000100:00100000:1.0:1769074788.591001:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947072 00000100:00100000:1.0:1769074788.591028:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14391 00000100:00100000:1.0:1769074788.591035:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2407:x1855007616947072:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074788.591057:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074788.591071:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd54655 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2e4 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074788.591106:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625191040:192.168.204.60@tcp:106: 00000100:00100000:1.0:1769074788.591134:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074788.591171:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.591656:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946944 00000100:00100000:0.0:1769074788.591661:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946944 00000100:00100000:0.0:1769074788.591702:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946432 00000100:00100000:0.0:1769074788.591706:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946432 00000100:00100000:0.0:1769074788.591728:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8722 00000100:00100000:0.0:1769074788.591735:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616946944:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074788.591757:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.591769:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074788.591771:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074788.591774:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.591783:0:28434:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.591831:0:24031:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8723 00000100:00100000:0.0:1769074788.591839:0:24031:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2408:x1855007616946432:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074788.591856:0:24031:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.591868:0:24031:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 28 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.591880:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074788.591882:0:24031:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074788.591885:0:24031:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.591907:0:24031:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 28 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.591917:0:24031:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 28 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074788.591930:0:24031:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d500 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_009:lustre-OST0000_UUID:24031:1855007625191168:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074788.591984:0:24031:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074788.592186:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616946560 00000100:00100000:2.0:1769074788.592192:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616946560 00010000:00010000:2.0:1769074788.592216:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 6/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 37 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:2.0:1769074788.592251:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7162 00000100:00100000:2.0:1769074788.592259:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2406:x1855007616946560:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074788.592268:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074788.592287:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 6/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 37 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.592299:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 37 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.592318:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.592320:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.592324:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.592332:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.592347:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074788.592377:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2406:x1855007616946560:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 119us (2223us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.592385:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7162 00000100:00100000:2.0:1769074788.592452:0:16436:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273b80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_003:lustre-OST0000_UUID:16436:1855007625190528:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074788.592470:0:16436:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.593062:0:28444:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_012:lustre-OST0001_UUID:28444:1855007625190656:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.593078:0:28444:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.593104:0:28448:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc380 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_015:lustre-OST0001_UUID:28448:1855007625190400:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074788.593116:0:28448:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.593124:0:28448:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.593131:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.604748:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.604799:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074788.605631:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947200 00000100:00100000:2.0:1769074788.605644:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947200 00010000:00010000:2.0:1769074788.605686:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:2.0:1769074788.605730:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947328 00000100:00100000:2.0:1769074788.605734:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947328 00010000:00010000:2.0:1769074788.605748:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:2.0:1769074788.605769:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7163 00000100:00100000:2.0:1769074788.605777:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616947200:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074788.605792:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074788.605842:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.605856:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.605892:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866c00/0x7fe2323adfd545ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ac expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.605909:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.605912:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.605917:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.605927:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.605959:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074788.606015:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616947200:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 238us (1278us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.606023:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7163 00000100:00100000:2.0:1769074788.606060:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7164 00000100:00100000:2.0:1769074788.606068:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2409:x1855007616947328:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074788.606079:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074788.606091:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.606101:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.606115:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89800/0x7fe2323adfd545bb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2ba expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:2.0:1769074788.606125:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.606126:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.606129:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.606137:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.606152:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074788.606172:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2409:x1855007616947328:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 105us (1373us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.606179:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7164 00010000:00010000:2.0:1769074788.606204:0:16436:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.606212:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.609567:0:28444:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.609586:0:28444:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.612179:0:24031:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d500 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_009:lustre-OST0000_UUID:24031:1855007625191168:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074788.612209:0:24031:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.612265:0:8404:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0001_UUID:8404:1855007625190912:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074788.612281:0:8404:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.612316:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.612342:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947456 00000100:00100000:0.0:1769074788.612349:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947456 00010000:00010000:0.0:1769074788.612379:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074788.612423:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7165 00000100:00100000:0.0:1769074788.612433:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e665c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616947456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074788.612447:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074788.612468:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.612480:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.612501:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89600/0x7fe2323adfd545b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2b3 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.612516:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.612518:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.612522:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.612531:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.612553:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074788.612594:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e665c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2409:x1855007616947456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 162us (277us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.612602:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7165 00010000:00010000:0.0:1769074788.612655:0:24031:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.612664:0:24031:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.613244:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.613266:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947584 00000100:00100000:0.0:1769074788.613271:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947584 00010000:00010000:0.0:1769074788.613288:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074788.613310:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7166 00000100:00100000:0.0:1769074788.613317:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2409:x1855007616947584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074788.613326:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074788.613338:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.613348:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.613365:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd545c2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c1 expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.613376:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.613378:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.613381:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.613393:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.613400:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.613401:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.613404:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.613412:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.613433:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074788.613455:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2409:x1855007616947584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.613462:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7166 00010000:00010000:0.0:1769074788.613482:0:8404:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.613490:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.614099:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.614119:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947712 00000100:00100000:0.0:1769074788.614124:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947712 00010000:00010000:0.0:1769074788.614144:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074788.614167:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7167 00000100:00100000:0.0:1769074788.614175:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616947712:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074788.614183:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074788.614195:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.614206:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074788.614221:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.614222:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.614225:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.614233:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.614240:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.614241:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.614244:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.614251:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.614269:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074788.614290:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616947712:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 116us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.614297:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7167 00010000:00010000:0.0:1769074788.614316:0:28434:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.614325:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.615351:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd545c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2c8 expref: 29 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074788.615371:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625190784:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074788.615405:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.615414:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.615426:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.615978:0:8403:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.615988:0:8403:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.616924:0:8396:0:(ldlm_lockd.c:740:ldlm_handle_ast_error()) ### client (nid 192.168.204.60@tcp) returned -22 from glimpse AST (req@000000004f450f49 x1855007625191040) - normal race ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:3.0:1769074788.616952:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd545a6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2a5 expref: 38 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074788.616964:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625191040:192.168.204.60@tcp:106: 00010000:00010000:3.0:1769074788.616995:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd54655 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2e4 expref: 37 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.617012:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b641650a00/0x7fe2323adfd54655 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2e4 expref: 37 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.617021:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641650a00, rc 0) 00000100:00100000:3.0:1769074788.617076:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+36:2407:x1855007616947072:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 26040us (26098us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074788.617085:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14391 00010000:00010000:3.0:1769074788.617161:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.617172:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.617180:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.617187:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074788.617191:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.617199:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.617215:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.617221:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45a00/0x7fe2323adfd5462b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.617259:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2408:x1855007616945920:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 40880us (41320us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074788.617267:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8716 00000100:00100000:1.0:1769074788.631340:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.631393:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947840 00000100:00100000:3.0:1769074788.631403:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947840 00000100:00100000:3.0:1769074788.631485:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8724 00000100:00100000:3.0:1769074788.631495:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616947840:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074788.631544:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.631559:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.631561:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.631567:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.631581:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.667363:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.667466:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.667510:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.667530:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.667545:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074788.667577:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616947968 00000100:00100000:3.0:1769074788.667587:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616947968 00000100:00100000:3.0:1769074788.667630:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616948096 00000100:00100000:3.0:1769074788.667635:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616948096 00000100:00100000:3.0:1769074788.667652:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616948224 00000100:00100000:3.0:1769074788.667655:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616948224 00000100:00100000:3.0:1769074788.667669:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616948352 00000100:00100000:3.0:1769074788.667671:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616948352 00000100:00100000:3.0:1769074788.667684:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616948608 00000100:00100000:3.0:1769074788.667686:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616948608 00000100:00100000:3.0:1769074788.667703:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14392 00000100:00100000:3.0:1769074788.667713:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2409:x1855007616947968:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.667731:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.667754:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd54671 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2eb expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074788.667785:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bc:0x0] initial LVB size: 1048576, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x7e8 00010000:00010000:3.0:1769074788.667832:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.667835:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.667845:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd54671 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2eb expref: 38 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.667870:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd54671 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2eb expref: 38 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.667886:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446800, rc 0) 00000100:00100000:0.0:1769074788.668031:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14393 00000100:00100000:0.0:1769074788.668050:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616948096:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074788.668060:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074788.668070:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54678 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2f2 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.668083:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.668085:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.668090:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54678 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2f2 expref: 33 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074788.668102:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54678 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2f2 expref: 33 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074788.668112:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876a00, rc 0) 00000100:00100000:2.0:1769074788.668198:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14394 00000100:00100000:2.0:1769074788.668210:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616948224:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074788.668218:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074788.668227:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682164000/0x7fe2323adfd5467f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c2f9 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.668240:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.668241:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074788.668247:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164000/0x7fe2323adfd5467f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2f9 expref: 34 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074788.668259:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b682164000/0x7fe2323adfd5467f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2f9 expref: 34 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074788.668268:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164000, rc 0) 00000100:00100000:1.0:1769074788.668353:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14395 00000100:00100000:1.0:1769074788.668363:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2407:x1855007616948352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074788.668370:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074788.668379:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54910c200/0x7fe2323adfd54686 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c300 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074788.668392:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074788.668394:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074788.668399:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54910c200/0x7fe2323adfd54686 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c300 expref: 39 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074788.668410:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54910c200/0x7fe2323adfd54686 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c300 expref: 39 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074788.668418:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54910c200, rc 0) 00000100:00100000:3.0:1769074788.670844:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+39:2409:x1855007616947968:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 3120us (13262us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.670857:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14392 00000100:00100000:3.0:1769074788.670900:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14396 00000100:00100000:3.0:1769074788.670913:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2407:x1855007616948608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074788.670932:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074788.670955:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5468d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c30e expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.670980:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.670982:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074788.670991:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5468d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c30e expref: 35 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.671007:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5468d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c30e expref: 35 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074788.671026:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547f0f000, rc 0) 00000100:00100000:0.0:1769074788.680426:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2407:x1855007616948096:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 12373us (12950us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.680441:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14393 00000100:00100000:0.0:1769074788.680866:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074788.680894:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616948480 00000100:00100000:0.0:1769074788.680903:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616948480 00000100:00100000:0.0:1769074788.680941:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14397 00000100:00100000:0.0:1769074788.680949:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+39:2407:x1855007616948480:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074788.680966:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074788.680984:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd54694 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c307 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.681009:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.681012:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074788.681022:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd54694 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c307 expref: 40 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074788.681049:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd54694 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c307 expref: 40 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074788.681064:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547876200, rc 0) 00000100:00100000:0.0:1769074788.681107:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+40:2407:x1855007616948480:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 159us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074788.681114:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14397 00000100:00100000:2.0:1769074788.692300:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616948224:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 24080us (24776us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074788.692314:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14394 00010000:00010000:2.0:1769074788.692417:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692431:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692440:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692449:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074788.692453:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692463:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692481:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692489:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866000/0x7fe2323adfd54632 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074788.692568:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2408:x1855007616946048:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 110229us (110590us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074788.692576:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8717 00010000:00010000:2.0:1769074788.692725:0:28444:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692735:0:28444:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692747:0:28444:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692752:0:28444:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba44000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074788.692756:0:28444:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692763:0:28444:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692773:0:28444:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.692780:0:28444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44000/0x7fe2323adfd54639 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074788.692846:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+38:2408:x1855007616946176:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 109768us (110803us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074788.692853:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8718 00000100:00100000:1.0:1769074788.693638:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:2407:x1855007616948352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 25273us (26103us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074788.693648:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14395 00000100:00100000:3.0:1769074788.709504:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2407:x1855007616948608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 38583us (41944us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074788.709518:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14396 00010000:00010000:0.0:1769074788.713742:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.713763:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.713772:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.713780:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b679924c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074788.713784:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.713799:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.713849:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.713868:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b679924c00/0x7fe2323adfd54647 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.713941:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616946304:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 123255us (123690us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074788.713952:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8720 00010000:00010000:0.0:1769074788.715633:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.715651:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.715660:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.715667:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074788.715671:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.715680:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.715702:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074788.715711:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877e00/0x7fe2323adfd5465c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074788.715794:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2406:x1855007616946944:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 124055us (124873us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074788.715804:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8722 00010000:00010000:2.0:1769074788.716571:0:24031:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.716587:0:24031:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.716596:0:24031:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.716603:0:24031:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074788.716607:0:24031:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.716615:0:24031:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.716629:0:24031:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.716636:0:24031:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876800/0x7fe2323adfd54663 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 24031 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074788.716704:0:24031:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616946432:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 124864us (125530us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074788.716712:0:24031:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8723 00010000:00010000:3.0:1769074788.723943:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.723962:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.723972:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.723979:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b679924400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074788.723983:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.723993:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.724010:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.724017:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b679924400/0x7fe2323adfd54640 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.724120:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007616946688:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 133683us (133907us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074788.724131:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8719 00010000:00010000:2.0:1769074788.736129:0:8403:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.736153:0:8403:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.736162:0:8403:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.736170:0:8403:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b679924600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074788.736176:0:8403:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.736185:0:8403:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.736202:0:8403:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074788.736209:0:8403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b679924600/0x7fe2323adfd5464e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8403 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074788.736292:0:8403:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2407:x1855007616946816:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 145480us (146017us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1769074788.736303:0:8403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8721 00010000:00010000:3.0:1769074788.738646:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.738668:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.738677:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.738683:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54ba45800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074788.738687:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.738696:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.738717:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074788.738724:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba45800/0x7fe2323adfd5466a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074788.738879:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616947840:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 107377us (107543us total) trans 0 rc 611424/611424 00000100:00100000:3.0:1769074788.738892:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8724 00010000:00010000:3.0:1769074788.766243:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074789.020643:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074789.020732:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616948864 00000100:00100000:3.0:1769074789.020744:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616948864 00000100:00100000:3.0:1769074789.020862:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4698 00000100:00100000:3.0:1769074789.020874:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63062:x1855007616948864:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074789.021116:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63062:x1855007616948864:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 244us (483us total) trans 8589944018 rc 0/0 00000100:00100000:3.0:1769074789.021124:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4698 00000100:00080000:1.0:1769074789.024482:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074789.063376:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074789.063501:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949120 00000100:00100000:2.0:1769074789.063512:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949120 00000100:00100000:2.0:1769074789.063576:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13376 00000100:00100000:2.0:1769074789.063586:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63065:x1855007616949120:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074789.063610:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074789.063630:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5469b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c315 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.063743:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd546a2 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074789.063758:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd546a2 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074789.063768:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd546a2 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074789.063780:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fb:0x0]: returning size 9000032 00010000:00010000:2.0:1769074789.063943:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5469b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c315 expref: 228 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.063963:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd546a2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c315 expref: 227 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.063972:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:2.0:1769074789.063975:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1769074789.063979:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074789.064054:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:63065:x1855007616949120:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 469us (687us total) trans 8589944019 rc 0/0 00000100:00100000:2.0:1769074789.064062:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13376 00000100:00100000:2.0:1769074789.121798:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074789.121992:0:28448:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949248 00000100:00100000:3.0:1769074789.122005:0:28448:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949248 00000100:00100000:3.0:1769074789.122109:0:28448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8725 00000100:00080000:1.0:1769074789.122112:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074789.122119:0:28448:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616949248:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074789.122160:0:28448:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.122179:0:28448:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.122187:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.122189:0:28448:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.122193:0:28448:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.122237:0:28448:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 25 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.122255:0:28448:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 25 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074789.122269:0:28448:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_015:lustre-OST0000_UUID:28448:1855007625191296:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074789.122361:0:28448:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074789.135456:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.135534:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.135569:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.135589:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.135627:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.135659:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949376 00000100:00100000:0.0:1769074789.135669:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949376 00000100:00100000:0.0:1769074789.135731:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949504 00000100:00100000:0.0:1769074789.135735:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949504 00000100:00100000:0.0:1769074789.135767:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949760 00000100:00100000:0.0:1769074789.135770:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949760 00000100:00100000:0.0:1769074789.135790:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8726 00000100:00100000:0.0:1769074789.135800:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616949376:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074789.135888:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.135908:0:28434:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.135917:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.135919:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.135923:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.135956:0:28434:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 26 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.135968:0:28434:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 26 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.135984:0:28434:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_010:lustre-OST0000_UUID:28434:1855007625191424:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074789.136073:0:28434:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee59600 going to sleep for 11 seconds 00000100:00100000:0.0:1769074789.136090:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950016 00000100:00100000:0.0:1769074789.136094:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950016 00000100:00100000:0.0:1769074789.136122:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8727 00000100:00100000:0.0:1769074789.136129:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616949504:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074789.136146:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136157:0:8404:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136164:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.136166:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.136169:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136190:0:8404:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136200:0:8404:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 33 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.136210:0:8404:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0001_UUID:8404:1855007625191552:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074789.136232:0:8404:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e7a6300 going to sleep for 11 seconds 00000100:00080000:0.0:1769074789.136244:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074789.136260:0:28444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8728 00000100:00100000:0.0:1769074789.136268:0:28444:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616949760:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074789.136282:0:28444:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136291:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.136293:0:28444:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.136295:0:28444:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136305:0:28444:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.136327:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8729 00000100:00100000:0.0:1769074789.136334:0:8402:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616950016:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074789.136349:0:8402:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136360:0:8402:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 33 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136366:0:8402:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.136368:0:8402:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.136370:0:8402:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136389:0:8402:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 33 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.136397:0:8402:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 33 pid: 27844 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.136406:0:8402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_000:lustre-OST0001_UUID:8402:1855007625191680:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074789.136429:0:8402:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4719c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074789.136616:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950272 00000100:00100000:1.0:1769074789.136621:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950272 00010000:00010000:1.0:1769074789.136643:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 27 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074789.136672:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7168 00000100:00100000:1.0:1769074789.136684:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616950272:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.136694:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.136713:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 27 pid: 36516 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.136724:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 27 pid: 36516 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.136745:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.136747:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.136751:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.136760:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 4 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.136774:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.136836:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2407:x1855007616950272:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 128us (1272us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.136845:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7168 00010000:00010000:3.0:1769074789.137017:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97400/0x7fe2323adfd5453d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c251 expref: 26 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074789.137052:0:28448:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_015:lustre-OST0000_UUID:28448:1855007625191296:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074789.137081:0:28448:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.137095:0:28448:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.137105:0:28448:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074789.153518:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.153580:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.153636:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.153790:0:8402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_000:lustre-OST0001_UUID:8402:1855007625191680:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074789.153864:0:8402:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074789.153925:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.153960:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.153991:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950528 00000100:00100000:1.0:1769074789.154000:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950528 00010000:00010000:1.0:1769074789.154053:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 34 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074789.154093:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950400 00000100:00100000:1.0:1769074789.154099:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950400 00010000:00010000:1.0:1769074789.154120:0:33102:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 27 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074789.154152:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7169 00000100:00100000:1.0:1769074789.154164:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616950528:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.154182:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.154207:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 34 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.154225:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 34 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.154254:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.154258:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.154263:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154274:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154284:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.154285:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.154288:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154296:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154329:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.154393:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616950528:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 230us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.154403:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7169 00000100:00100000:1.0:1769074789.154431:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7170 00000100:00100000:1.0:1769074789.154441:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616950400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.154454:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.154471:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 27 pid: 36516 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.154484:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 27 pid: 36516 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.154497:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.154499:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.154502:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154509:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154523:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.154553:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007616950400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 113us (593us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.154562:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7170 00010000:00010000:1.0:1769074789.154597:0:28444:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.154608:0:28444:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074789.155529:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.155567:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950656 00000100:00100000:1.0:1769074789.155577:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950656 00010000:00010000:1.0:1769074789.155609:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 35 pid: 27844 timeout: 1876 lvb_type: 0 00000100:00100000:1.0:1769074789.155644:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7171 00000100:00100000:1.0:1769074789.155656:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2406:x1855007616950656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.155669:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.155687:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 35 pid: 27844 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.155701:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 35 pid: 27844 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.155723:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b053000/0x7fe2323adfd54559 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c26d expref: 35 pid: 27844 timeout: 1876 lvb_type: 0 00010000:00010000:1.0:1769074789.155738:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.155740:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.155745:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.155754:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.155782:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.155879:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616950656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 224us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.155888:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7171 00010000:00010000:1.0:1769074789.155938:0:8402:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.155954:0:8402:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.157136:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949632 00000100:00100000:3.0:1769074789.157147:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949632 00000100:00100000:3.0:1769074789.157195:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616949888 00000100:00100000:3.0:1769074789.157199:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616949888 00000100:00100000:3.0:1769074789.157220:0:16436:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950144 00000100:00100000:3.0:1769074789.157224:0:16436:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950144 00000100:00100000:3.0:1769074789.157251:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8730 00000100:00100000:3.0:1769074789.157260:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616949632:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074789.157294:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.157307:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074789.157309:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074789.157313:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.157323:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.157875:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8731 00000100:00100000:3.0:1769074789.157883:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616949888:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074789.157902:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.157916:0:16437:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.157924:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.157925:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.157928:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.157959:0:16437:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 30 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.157973:0:16437:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 30 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074789.157990:0:16437:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0000_UUID:16437:1855007625191808:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074789.158062:0:16437:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260900 going to sleep for 11 seconds 00000100:00100000:3.0:1769074789.158081:0:28445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8732 00000100:00100000:3.0:1769074789.158089:0:28445:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616950144:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074789.158104:0:28445:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.158115:0:28445:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.158121:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.158122:0:28445:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.158124:0:28445:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.158140:0:28445:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 30 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.158150:0:28445:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 30 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074789.158159:0:28445:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625191936:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074789.158183:0:28445:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260780 going to sleep for 11 seconds 00010000:00010000:0.0:1769074789.158321:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b718000/0x7fe2323adfd5454b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c25f expref: 34 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.158337:0:8404:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0001_UUID:8404:1855007625191552:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074789.158359:0:8404:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.158368:0:8404:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.158378:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.158930:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d96e00/0x7fe2323adfd54544 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c258 expref: 30 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.158942:0:28434:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_010:lustre-OST0000_UUID:28434:1855007625191424:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074789.158957:0:28434:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.158965:0:28434:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.158971:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.207081:0:28445:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462df100 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_013:lustre-OST0000_UUID:28445:1855007625191936:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074789.207127:0:28445:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.208117:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.208163:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950912 00000100:00100000:0.0:1769074789.208175:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950912 00010000:00010000:0.0:1769074789.208221:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 31 pid: 8398 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.208277:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7172 00000100:00100000:0.0:1769074789.208290:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e667450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616950912:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074789.208309:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074789.208335:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 31 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074789.208349:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 31 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074789.208373:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54b718600/0x7fe2323adfd54560 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c274 expref: 31 pid: 8398 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074789.208390:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.208391:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.208396:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.208406:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.208433:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074789.208486:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e667450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007616950912:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 200us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.208494:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7172 00010000:00010000:0.0:1769074789.208556:0:28445:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.208570:0:28445:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.208989:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.209010:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616950784 00000100:00100000:0.0:1769074789.209015:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616950784 00010000:00010000:0.0:1769074789.209046:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 31 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:0.0:1769074789.209069:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7173 00000100:00100000:0.0:1769074789.209077:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616950784:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074789.209086:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074789.209099:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 31 pid: 36516 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074789.209111:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 31 pid: 36516 timeout: 1876 lvb_type: 0 00010000:00010000:0.0:1769074789.209124:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.209126:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.209130:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.209137:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.209150:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074789.209171:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2408:x1855007616950784:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 95us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.209177:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7173 00010000:00010000:1.0:1769074789.210174:0:28444:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210196:0:28444:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210206:0:28444:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210216:0:28444:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074789.210221:0:28444:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210233:0:28444:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210257:0:28444:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210266:0:28444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876000/0x7fe2323adfd546c5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28444 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074789.210358:0:28444:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616949760:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 74087us (74764us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074789.210371:0:28444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8728 00010000:00010000:1.0:1769074789.210427:0:8402:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210437:0:8402:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210446:0:8402:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210454:0:8402:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547876400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074789.210459:0:8402:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210467:0:8402:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210481:0:8402:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.210489:0:8402:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876400/0x7fe2323adfd546cc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8402 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074789.210525:0:8402:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616950016:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 74191us (74897us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074789.210532:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8729 00010000:00010000:3.0:1769074789.213780:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547d97800/0x7fe2323adfd54552 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c266 expref: 30 pid: 36516 timeout: 1876 lvb_type: 0 00000100:00100000:3.0:1769074789.213852:0:16437:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0000_UUID:16437:1855007625191808:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074789.213881:0:16437:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.213893:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.213900:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214551:0:28448:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214563:0:28448:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214571:0:28448:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214578:0:28448:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074789.214582:0:28448:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214589:0:28448:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214610:0:28448:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.214618:0:28448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9200/0x7fe2323adfd546b0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28448 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.214678:0:28448:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007616949248:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 92559us (92891us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074789.214689:0:28448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8725 00010000:00010000:3.0:1769074789.219647:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.219665:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.219674:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.219681:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074789.219685:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.219694:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.219713:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.219720:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8000/0x7fe2323adfd546d3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.219836:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2408:x1855007616949632:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 62572us (66325us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074789.219847:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8730 00010000:00010000:0.0:1769074789.237282:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.237304:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.237313:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.237320:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074789.237324:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.237333:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.237372:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.237380:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547877400/0x7fe2323adfd546be lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.237496:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007616949504:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 101360us (101919us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074789.237508:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8727 00010000:00010000:1.0:1769074789.251441:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.251465:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.251476:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.251483:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547877800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074789.251487:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.251495:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.251515:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.251522:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877800/0x7fe2323adfd546b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074789.251629:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616949376:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 115825us (116189us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1769074789.251644:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8726 00010000:00010000:2.0:1769074789.253047:0:28445:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.253064:0:28445:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.253073:0:28445:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.253080:0:28445:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c8400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074789.253085:0:28445:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.253099:0:28445:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.253119:0:28445:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.253127:0:28445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c8400/0x7fe2323adfd546e1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28445 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074789.253222:0:28445:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007616950144:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 95129us (99583us total) trans 0 rc 611424/611424 00000100:00100000:2.0:1769074789.253232:0:28445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8732 00010000:00010000:3.0:1769074789.263242:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.263268:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.263279:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.263287:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074789.263293:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.263304:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.263330:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.263340:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5495c9c00/0x7fe2323adfd546da lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.263493:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616949888:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 105605us (109908us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074789.263508:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8731 00000100:00100000:0.0:1769074789.277178:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625192064:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074789.277265:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074789.277317:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625192064 00000100:00100000:0.0:1769074789.277324:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625192064 00000100:00100000:2.0:1769074789.277543:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2656 00000100:00100000:2.0:1769074789.277556:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625192064:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074789.277593:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445400576 avail 1299161088 00000100:00100000:2.0:1769074789.277639:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625192064:12345-0@lo:41:osp-pre-0-1.0 Request processed in 85us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074789.277645:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2656 00000100:00100000:1.0:1769074789.277974:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625192192:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074789.278003:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074789.279155:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625192064:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074789.280324:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625192192 00000100:00100000:3.0:1769074789.280336:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625192192 00000100:00100000:1.0:1769074789.280631:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2657 00000100:00100000:1.0:1769074789.280641:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625192192:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074789.280677:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446150144 avail 1299910656 00000100:00100000:1.0:1769074789.280756:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625192192:12345-0@lo:41:osp-pre-1-0.0 Request processed in 117us (2752us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.280763:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2657 00000100:00100000:3.0:1769074789.280846:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625192192:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074789.385551:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.385602:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.385638:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.385658:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.385675:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.385701:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951040 00000100:00100000:0.0:1769074789.385712:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951040 00000100:00100000:0.0:1769074789.385758:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951168 00000100:00100000:0.0:1769074789.385761:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951168 00000100:00100000:0.0:1769074789.385776:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951296 00000100:00100000:0.0:1769074789.385778:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951296 00000100:00100000:0.0:1769074789.385792:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951424 00000100:00100000:0.0:1769074789.385795:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951424 00000100:00100000:0.0:1769074789.385839:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951552 00000100:00100000:0.0:1769074789.385842:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951552 00000100:00100000:0.0:1769074789.385878:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14398 00000100:00100000:0.0:1769074789.385886:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616951040:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074789.385904:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074789.385924:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd546e8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c31c expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074789.385955:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2b9:0x0] initial LVB size: 2097152, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0xfe8 00010000:00010000:0.0:1769074789.385979:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.385983:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.385996:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd546e8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c31c expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386024:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd546e8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c31c expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386191:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cdeaa00, rc 0) 00000100:00100000:0.0:1769074789.386275:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616951040:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 386us (730us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.386285:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14398 00000100:00100000:0.0:1769074789.386316:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14399 00000100:00100000:0.0:1769074789.386327:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616951168:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074789.386338:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074789.386354:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea400/0x7fe2323adfd546ef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c323 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074789.386394:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bd:0x0] initial LVB size: 3145728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x1000 00010000:00010000:0.0:1769074789.386409:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386411:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386419:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea400/0x7fe2323adfd546ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c323 expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386433:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea400/0x7fe2323adfd546ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c323 expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386444:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cdea400, rc 0) 00000100:00100000:0.0:1769074789.386475:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616951168:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 150us (871us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.386484:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14399 00000100:00100000:0.0:1769074789.386496:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14400 00000100:00100000:0.0:1769074789.386503:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616951296:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074789.386513:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074789.386528:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeba00/0x7fe2323adfd546f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c32a expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074789.386544:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c0:0x0] initial LVB size: 3145728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x1000 00010000:00010000:0.0:1769074789.386557:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386559:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386566:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeba00/0x7fe2323adfd546f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c32a expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386580:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeba00/0x7fe2323adfd546f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c32a expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386589:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cdeba00, rc 0) 00000100:00100000:0.0:1769074789.386620:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616951296:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 116us (980us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.386627:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14400 00000100:00100000:0.0:1769074789.386641:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14401 00000100:00100000:0.0:1769074789.386648:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007616951424:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074789.386654:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074789.386663:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea800/0x7fe2323adfd546fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c331 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074789.386679:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2be:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:0.0:1769074789.386687:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386688:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386694:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea800/0x7fe2323adfd546fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c331 expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386702:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea800/0x7fe2323adfd546fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c331 expref: 24 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386709:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cdea800, rc 0) 00000100:00100000:0.0:1769074789.386727:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616951424:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 80us (1068us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.386733:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14401 00000100:00100000:0.0:1769074789.386744:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14402 00000100:00100000:0.0:1769074789.386751:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2409:x1855007616951552:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074789.386758:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074789.386766:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54cdea600/0x7fe2323adfd54704 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c338 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074789.386777:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c1:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:0.0:1769074789.386784:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386786:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.386791:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdea600/0x7fe2323adfd54704 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c338 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386800:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b54cdea600/0x7fe2323adfd54704 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c338 expref: 31 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.386840:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cdea600, rc 0) 00000100:00100000:0.0:1769074789.386859:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2409:x1855007616951552:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 109us (1183us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.386864:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14402 00000100:00100000:0.0:1769074789.386963:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.386987:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951680 00000100:00100000:0.0:1769074789.386993:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951680 00000100:00100000:0.0:1769074789.387019:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14403 00000100:00100000:0.0:1769074789.387025:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616951680:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074789.387044:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074789.387055:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea200/0x7fe2323adfd5470b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c33f expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074789.387067:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bf:0x0] initial LVB size: 4805728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x4b0 00010000:00010000:0.0:1769074789.387076:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.387077:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074789.387082:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea200/0x7fe2323adfd5470b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c33f expref: 25 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.387093:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea200/0x7fe2323adfd5470b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c33f expref: 25 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074789.387101:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b54cdea200, rc 0) 00000100:00100000:0.0:1769074789.387121:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616951680:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 96us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.387130:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14403 00000100:00100000:1.0:1769074789.503563:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074789.503798:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951808 00000100:00100000:3.0:1769074789.503863:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951808 00000100:00100000:3.0:1769074789.503954:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8733 00000100:00100000:3.0:1769074789.503964:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2407:x1855007616951808:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074789.504001:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.504022:0:16437:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.504040:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.504042:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074789.504046:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.504082:0:16437:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 31 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.504097:0:16437:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 31 pid: 8398 timeout: 1877 lvb_type: 0 00000100:00100000:3.0:1769074789.504111:0:16437:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625192320:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074789.504232:0:16437:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64b260900 going to sleep for 11 seconds 00000100:00100000:0.0:1769074789.510618:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.510670:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.510765:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952064 00000100:00100000:1.0:1769074789.510776:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952064 00000100:00100000:1.0:1769074789.510885:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616951936 00000100:00100000:1.0:1769074789.510889:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616951936 00000100:00100000:1.0:1769074789.510924:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8734 00000100:00100000:1.0:1769074789.510934:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616952064:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074789.510973:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.510992:0:28434:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.511002:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074789.511004:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074789.511008:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.511054:0:28434:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 33 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.511067:0:28434:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 33 pid: 8398 timeout: 1877 lvb_type: 0 00000100:00100000:1.0:1769074789.511088:0:28434:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_010:lustre-OST0001_UUID:28434:1855007625192448:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074789.511174:0:28434:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643510180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074789.511520:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8735 00000100:00100000:0.0:1769074789.511531:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2406:x1855007616951936:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074789.511552:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.511565:0:8404:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 33 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.511572:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.511573:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074789.511576:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 4/1,0 mode: --/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.511596:0:8404:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 33 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.511605:0:8404:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 33 pid: 27844 timeout: 1877 lvb_type: 0 00000100:00100000:0.0:1769074789.511615:0:8404:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0001_UUID:8404:1855007625192576:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074789.511657:0:8404:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4719c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074789.524513:0:28434:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_010:lustre-OST0001_UUID:28434:1855007625192448:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074789.524545:0:28434:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.524635:0:16437:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_004:lustre-OST0001_UUID:16437:1855007625192320:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074789.524652:0:16437:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.524691:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074789.524739:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.524839:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952192 00000100:00100000:1.0:1769074789.524855:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952192 00010000:00010000:1.0:1769074789.524887:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 34 pid: 8398 timeout: 1877 lvb_type: 0 00000100:00100000:1.0:1769074789.524918:0:6499:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952320 00000100:00100000:1.0:1769074789.524921:0:6499:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952320 00010000:00010000:1.0:1769074789.524931:0:6499:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 35 pid: 8398 timeout: 1877 lvb_type: 0 00000100:00100000:1.0:1769074789.524955:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7174 00000100:00100000:1.0:1769074789.524962:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616952192:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.524976:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.524995:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 35 pid: 8398 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.525007:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 35 pid: 8398 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.525045:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54b719800/0x7fe2323adfd54583 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c289 expref: 35 pid: 8398 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.525060:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.525062:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.525066:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.525076:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.525095:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.525136:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2409:x1855007616952192:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 174us (769us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.525143:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7174 00000100:00100000:1.0:1769074789.525158:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7175 00000100:00100000:1.0:1769074789.525164:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007616952320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.525170:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.525179:0:6499:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 33 pid: 8398 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.525188:0:6499:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 33 pid: 8398 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.525200:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867400/0x7fe2323adfd54591 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c297 expref: 33 pid: 8398 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.525213:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.525215:0:6499:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.525218:0:6499:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.525225:0:6499:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.525238:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.525252:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007616952320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 89us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.525258:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7175 00010000:00010000:1.0:1769074789.525273:0:28434:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.525281:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.525572:0:16437:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.525581:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.554254:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074789.554396:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952448 00000100:00100000:1.0:1769074789.554407:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952448 00010000:00010000:1.0:1769074789.554457:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 34 pid: 27844 timeout: 1877 lvb_type: 0 00000100:00100000:1.0:1769074789.554508:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7176 00000100:00100000:1.0:1769074789.554518:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616952448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074789.554532:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074789.554553:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 34 pid: 27844 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.554567:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 34 pid: 27844 timeout: 1877 lvb_type: 0 00010000:00010000:1.0:1769074789.554592:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.554594:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074789.554599:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.554609:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074789.554644:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074789.554705:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2406:x1855007616952448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 188us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.554713:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7176 00010000:00010000:2.0:1769074789.555413:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88600/0x7fe2323adfd5458a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c290 expref: 33 pid: 27844 timeout: 1877 lvb_type: 0 00000100:00100000:2.0:1769074789.555435:0:8404:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6680 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0001_UUID:8404:1855007625192576:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074789.555483:0:8404:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.555494:0:8404:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074789.555500:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628753:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628778:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628788:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628796:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b5495c9c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074789.628801:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628858:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628883:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.628891:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5495c9c00/0x7fe2323adfd54712 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.629087:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2407:x1855007616951808:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 125117us (125529us total) trans 0 rc 4194304/4194304 00000100:00100000:0.0:1769074789.629100:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8733 00010000:00010000:3.0:1769074789.632094:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.632113:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.632122:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.632130:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074789.632134:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.632144:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.632167:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074789.632174:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54719 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [4194304->4808703] (req 4194304->4808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074789.632276:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616952064:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 121338us (121663us total) trans 0 rc 611424/611424 00000100:00100000:3.0:1769074789.632287:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8734 00010000:00010000:0.0:1769074789.664130:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/1,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.664159:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.664169:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.664176:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b54cdebc00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074789.664182:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.664192:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.664216:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074789.664226:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cdebc00/0x7fe2323adfd54720 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074789.664361:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:2406:x1855007616951936:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 152828us (153685us total) trans 0 rc 4194304/4194304 00000100:00100000:0.0:1769074789.664381:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8735 00000100:00100000:3.0:1769074789.789086:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625192704:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074789.789151:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074789.789213:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625192704 00000100:00100000:3.0:1769074789.789220:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625192704 00000100:00100000:3.0:1769074789.789274:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1176 00000100:00100000:3.0:1769074789.789281:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625192704:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074789.789308:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3905998848 avail 3674476544 00000100:00100000:3.0:1769074789.789351:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625192704:12345-0@lo:13:osp-pre-0-1.0 Request processed in 69us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074789.789364:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1176 00000100:00080000:3.0:1769074789.789395:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074789.789432:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625192704:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074789.790188:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074789.790204:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625192832:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074789.790247:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074789.790362:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625192832 00000100:00100000:1.0:1769074789.790379:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625192832 00000100:00100000:1.0:1769074789.790433:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1177 00000100:00100000:1.0:1769074789.790440:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625192832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074789.790454:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074789.790496:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625192832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074789.790502:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1177 00000100:00100000:1.0:1769074789.790532:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625192832:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1769074789.790611:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074789.790935:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074789.790946:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625192960:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074789.790978:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074789.791060:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625192960 00000100:00100000:0.0:1769074789.791065:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625192960 00000100:00100000:0.0:1769074789.791106:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1178 00000100:00100000:0.0:1769074789.791114:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625192960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074789.791132:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3900067840 avail 3668545536 00000100:00100000:0.0:1769074789.791166:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625192960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074789.791172:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1178 00000100:00100000:0.0:1769074789.791205:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625192960:0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1769074789.791261:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074789.791344:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074790.000926:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074790.000976:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074790.001015:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074790.001194:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952576 00000100:00100000:1.0:1769074790.001206:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952576 00000100:00100000:1.0:1769074790.001262:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952832 00000100:00100000:1.0:1769074790.001265:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952832 00000100:00100000:1.0:1769074790.001279:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952704 00000100:00100000:1.0:1769074790.001281:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952704 00000100:00100000:1.0:1769074790.001297:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14404 00000100:00100000:1.0:1769074790.001304:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616952576:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074790.001323:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074790.001344:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54727 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c346 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074790.001367:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bd:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1fe8 00010000:00010000:1.0:1769074790.001385:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074790.001387:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074790.001396:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54727 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c346 expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074790.001412:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54727 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c346 expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074790.001423:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:1.0:1769074790.001471:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616952576:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 168us (572us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074790.001496:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14404 00000100:00100000:1.0:1769074790.001611:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14405 00000100:00100000:1.0:1769074790.001617:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616952832:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074790.001624:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074790.001633:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd5472e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c354 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074790.001646:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c4:0x0] initial LVB size: 4805728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x4b0 00010000:00010000:1.0:1769074790.001655:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074790.001657:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074790.001662:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd5472e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c354 expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074790.001671:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd5472e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c354 expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074790.001679:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:1.0:1769074790.001699:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616952832:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 82us (723us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074790.001705:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14405 00000100:00100000:1.0:1769074790.001717:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14406 00000100:00100000:1.0:1769074790.001723:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2406:x1855007616952704:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074790.001729:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074790.001737:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54735 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c34d expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074790.001748:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c3:0x0] initial LVB size: 8388608, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x2000 00010000:00010000:1.0:1769074790.001757:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074790.001758:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074790.001763:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54735 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c34d expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074790.001772:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54735 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c34d expref: 31 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074790.001781:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba400, rc 0) 00000100:00100000:1.0:1769074790.001801:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2406:x1855007616952704:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 78us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074790.001849:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14406 00000100:00100000:2.0:1769074790.301391:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625193088:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074790.301463:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074790.302168:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193088 00000100:00100000:1.0:1769074790.302177:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193088 00000100:00100000:1.0:1769074790.302244:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1179 00000100:00100000:1.0:1769074790.302253:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625193088:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1769074790.302275:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3900067840 avail 3668545536 00000100:00100000:1.0:1769074790.302334:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625193088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (871us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074790.302341:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1179 00000100:00100000:2.0:1769074790.302490:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625193088:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1769074790.302729:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00100000:1.0:1769074790.304728:0:7431:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0001: committing for initial connect of 0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000001:00080000:1.0:1769074790.304743:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968347 is committed 00000020:00000080:1.0:1769074790.304749:0:7431:0:(genops.c:904:class_export_put()) final put ffff89b5452d5800/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b 00000001:00100000:1.0:1769074790.304776:0:7431:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0001: committing for initial connect of 543044d2-7a16-4e58-9b2c-28bb782e5779 00000001:00080000:1.0:1769074790.304781:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968346 is committed 00000020:00000080:1.0:1769074790.304784:0:7431:0:(genops.c:904:class_export_put()) final put ffff89b649250800/543044d2-7a16-4e58-9b2c-28bb782e5779 00000020:00000080:2.0:1769074790.304882:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b5452d5800/0bbd97bd-31b2-46f4-8b1b-d137c4ecd83b for lustre-MDT0001 00000020:00000080:2.0:1769074790.304952:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b649250800/543044d2-7a16-4e58-9b2c-28bb782e5779 for lustre-MDT0001 00000020:00000080:2.0:1769074790.304982:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b5452d0800 for lustre-MDT0001 00000020:00000080:2.0:1769074790.304987:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b649256800 for lustre-MDT0001 00010000:00010000:0.0:1769074790.813125:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074790.855045:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074790.855081:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074790.855121:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616953216 00000100:00100000:2.0:1769074790.855134:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616953216 00000100:00100000:2.0:1769074790.855218:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2348 00000100:00100000:2.0:1769074790.855235:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616953216:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074790.855320:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616953216:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 92us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074790.855331:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2348 00000100:00100000:2.0:1769074790.855573:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:1.0:1769074790.855617:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074790.855640:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616953472 00000100:00100000:1.0:1769074790.855645:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616953472 00000100:00100000:1.0:1769074790.855680:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1466 00000100:00100000:1.0:1769074790.855688:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2407:x1855007616953472:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074790.855722:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:2407:x1855007616953472:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 34us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074790.855733:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1466 00000100:00100000:2.0:1769074790.857226:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074790.857332:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074790.857377:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616953344 00000100:00100000:2.0:1769074790.857385:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616953344 00000100:00100000:2.0:1769074790.857428:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1467 00000100:00100000:2.0:1769074790.857447:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616953344:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074790.857508:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2407:x1855007616953344:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 71us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074790.857518:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1467 00000100:00100000:1.0:1769074790.859773:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616953088 00000100:00100000:1.0:1769074790.859789:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616953088 00000100:00100000:1.0:1769074790.859906:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2349 00000100:00100000:1.0:1769074790.859922:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2407:x1855007616953088:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074790.860025:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+227:2407:x1855007616953088:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 109us (2808us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074790.860033:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2349 00000100:00100000:3.0:1769074790.862081:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616952960 00000100:00100000:3.0:1769074790.862094:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616952960 00000100:00100000:3.0:1769074790.862186:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 337 00000100:00100000:3.0:1769074790.862196:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007616952960:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074790.862273:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007616952960:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 76us (7237us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074790.862283:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 337 00010000:00010000:3.0:1769074791.839555:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074792.350208:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625193344:0@lo:400:kworker.0 00000100:00100000:1.0:1769074792.350235:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625193472:0@lo:400:kworker.0 00000100:00100000:0.0:1769074792.350337:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625193216:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.350407:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625193600:0@lo:400:kworker.0 00000100:00100000:1.0:1769074792.353158:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074792.353158:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074792.353236:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074792.353372:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193472 00000100:00100000:0.0:1769074792.353383:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193472 00000100:00100000:0.0:1769074792.353440:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2350 00000100:00100000:0.0:1769074792.353450:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625193472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074792.353509:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625193472:12345-0@lo:400:kworker.0 Request processed in 61us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074792.353516:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2350 00000100:00100000:0.0:1769074792.353545:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193216 00000100:00100000:0.0:1769074792.353549:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193216 00000100:00100000:0.0:1769074792.353572:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:3.0:1769074792.353573:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625193856:0@lo:400:kworker.0 00000100:00100000:0.0:1769074792.353578:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625193216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.353605:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074792.353606:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625193216:12345-0@lo:400:kworker.0 Request processed in 28us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074792.353612:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:3.0:1769074792.353630:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625194368:0@lo:400:kworker.0 00000100:00100000:0.0:1769074792.353644:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193856 00000100:00100000:0.0:1769074792.353647:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193856 00000100:00100000:3.0:1769074792.353651:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074792.353675:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2351 00000100:00100000:3.0:1769074792.353680:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193344 00000100:00100000:0.0:1769074792.353682:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625193856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.353683:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193344 00000100:00100000:0.0:1769074792.353712:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625193856:12345-0@lo:400:kworker.0 Request processed in 31us (106us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074792.353718:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2351 00000100:00080000:0.0:1769074792.353738:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074792.353891:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625193472:0@lo:400:kworker.0 00000100:00100000:1.0:1769074792.353900:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625193984:0@lo:400:kworker.0 00000100:00100000:1.0:1769074792.353925:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074792.354021:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074792.354051:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625194112:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354079:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193600 00000100:00100000:2.0:1769074792.354083:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193600 00000100:00100000:2.0:1769074792.354116:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1468 00000100:00100000:2.0:1769074792.354123:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625193600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354158:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625193600:12345-0@lo:400:kworker.0 Request processed in 36us (136us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074792.354163:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1468 00000100:00080000:2.0:1769074792.354182:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074792.354205:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194368 00000100:00100000:3.0:1769074792.354209:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194368 00000100:00100000:1.0:1769074792.354244:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625194496:0@lo:400:kworker.0 00000100:00100000:1.0:1769074792.354267:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074792.354347:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194496 00000100:00100000:2.0:1769074792.354352:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194496 00000100:00100000:2.0:1769074792.354390:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1469 00000100:00100000:2.0:1769074792.354395:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625194496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354434:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+13:3650:x1855007625194496:12345-0@lo:400:kworker.0 Request processed in 38us (165us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074792.354439:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1469 00000100:00100000:3.0:1769074792.354454:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2658 00000100:00100000:2.0:1769074792.354457:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074792.354461:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625193344:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1769074792.354492:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074792.354496:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625193344:12345-0@lo:400:kworker.0 Request processed in 36us (1348us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074792.354503:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2658 00000100:00100000:3.0:1769074792.354515:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2659 00000100:00100000:3.0:1769074792.354521:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625194368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.354547:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625194368:12345-0@lo:400:kworker.0 Request processed in 26us (894us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074792.354556:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2659 00000100:00100000:2.0:1769074792.354593:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625194624:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354622:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074792.354651:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625193600:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354670:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194624 00000100:00100000:2.0:1769074792.354672:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194624 00000100:00100000:2.0:1769074792.354697:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1470 00000100:00100000:2.0:1769074792.354702:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625194624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354729:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+14:3651:x1855007625194624:12345-0@lo:400:kworker.0 Request processed in 27us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074792.354734:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1470 00000100:00080000:2.0:1769074792.354744:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074792.354759:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625194624:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354777:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625194496:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354884:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625193216:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354894:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625193728:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354920:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074792.354936:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625194240:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.354955:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074792.355019:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193984 00000100:00100000:0.0:1769074792.355024:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193984 00000100:00100000:0.0:1769074792.355055:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194112 00000100:00100000:0.0:1769074792.355058:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194112 00000100:00100000:0.0:1769074792.355080:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2352 00000100:00100000:3.0:1769074792.355084:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2353 00000100:00100000:0.0:1769074792.355086:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625193984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.355090:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625194112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074792.355124:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625193984:12345-0@lo:400:kworker.0 Request processed in 38us (1199us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074792.355124:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625194112:12345-0@lo:400:kworker.0 Request processed in 33us (664us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074792.355130:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2353 00000100:00100000:0.0:1769074792.355137:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2352 00000100:00100000:3.0:1769074792.355160:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625194368:0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.355171:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625193344:0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.355181:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625193856:0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.355208:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625194112:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.355237:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625194752:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.355262:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074792.355279:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194240 00000100:00100000:1.0:1769074792.355283:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194240 00000100:00100000:0.0:1769074792.355320:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2354 00000100:00100000:0.0:1769074792.355328:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625194240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074792.355357:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625194240:12345-0@lo:400:kworker.0 Request processed in 31us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074792.355362:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2354 00000100:00100000:2.0:1769074792.355478:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625194240:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.355496:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194752 00000100:00100000:2.0:1769074792.355500:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194752 00000100:00100000:2.0:1769074792.355526:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2355 00000100:00100000:2.0:1769074792.355532:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625194752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.355569:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625194752:12345-0@lo:400:kworker.0 Request processed in 37us (306us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074792.355575:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2355 00000100:00100000:2.0:1769074792.355600:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625194752:0@lo:400:kworker.0 00000100:00100000:2.0:1769074792.355624:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625193984:0@lo:400:kworker.0 00000100:00080000:3.0:1769074792.357667:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074792.357678:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074792.357707:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625193728 00000100:00100000:3.0:1769074792.357718:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625193728 00000100:00100000:3.0:1769074792.357969:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1471 00000100:00100000:3.0:1769074792.357990:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625193728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074792.358086:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625193728:12345-0@lo:400:kworker.0 Request processed in 108us (3166us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074792.358094:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1471 00000100:00100000:3.0:1769074792.358142:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625193728:0@lo:400:kworker.0 00000100:00080000:2.0:1769074792.363414:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074792.861788:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074793.887635:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1769074793.893846:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944019 is committed 00000001:00080000:0.0:1769074793.895262:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874596 is committed 00000001:00080000:0.0:1769074793.895294:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874587 is committed 00000001:00080000:2.0:1769074793.895386:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841264 is committed 00000001:00080000:2.0:1769074793.895401:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841256 is committed 00000100:00100000:3.0:1769074794.398014:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625194880:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074794.398092:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074794.398150:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625194880 00000100:00100000:3.0:1769074794.398160:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625194880 00000100:00100000:3.0:1769074794.398210:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2660 00000100:00100000:3.0:1769074794.398219:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625194880:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074794.398254:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074794.400940:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625194880:12345-0@lo:41:osp-pre-1-0.0 Request processed in 2715us (2841us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074794.400968:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2660 00000100:00100000:3.0:1769074794.401037:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625194880:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074794.401063:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074794.401073:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625195008:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074794.401124:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074794.417186:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195008 00000100:00100000:0.0:1769074794.417206:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195008 00000100:00100000:0.0:1769074794.417281:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2661 00000100:00100000:0.0:1769074794.417582:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625195008:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074794.417627:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445400576 avail 1299161088 00000100:00100000:0.0:1769074794.417713:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625195008:12345-0@lo:41:osp-pre-0-1.0 Request processed in 422us (16588us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074794.417720:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2661 00000100:00100000:0.0:1769074794.417781:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625195008:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1769074794.909424:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074794.910405:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625195136:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074794.910494:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074794.910583:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1769074794.910590:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625195264:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074794.910614:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074794.910782:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195136 00000100:00100000:1.0:1769074794.910792:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195136 00000100:00100000:1.0:1769074794.910873:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195264 00000100:00100000:1.0:1769074794.910877:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195264 00000100:00100000:1.0:1769074794.910901:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1180 00000100:00100000:1.0:1769074794.910910:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625195136:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074794.910951:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074794.911003:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625195136:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (509us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074794.911010:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1180 00000100:00100000:1.0:1769074794.911029:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1181 00000100:00100000:1.0:1769074794.911035:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625195264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074794.911049:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3900067840 avail 3668545536 00000100:00100000:1.0:1769074794.911076:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625195264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 41us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074794.911081:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1181 00000100:00100000:1.0:1769074794.911125:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874596 gen 2 00000004:00080000:1.0:1769074794.911130:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d101180, transno 17179874588 00000004:00080000:1.0:1769074794.911137:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d100700, transno 17179874589 00000004:00080000:1.0:1769074794.911140:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d100000, transno 17179874590 00000004:00080000:1.0:1769074794.911142:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d102a00, transno 17179874591 00000004:00080000:1.0:1769074794.911144:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d101500, transno 17179874592 00000004:00080000:1.0:1769074794.911147:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d100a80, transno 17179874593 00000004:00080000:1.0:1769074794.911149:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d103100, transno 17179874594 00000004:00080000:1.0:1769074794.911152:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a271880, transno 17179874595 00000004:00080000:1.0:1769074794.911154:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d101c00, transno 17179874596 00000100:00100000:1.0:1769074794.911165:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625195264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074794.911174:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841264 gen 2 00000004:00080000:1.0:1769074794.911176:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef5500, transno 21474841257 00000004:00080000:1.0:1769074794.911183:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475ddc00, transno 21474841258 00000004:00080000:1.0:1769074794.911185:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54b425180, transno 21474841259 00000004:00080000:1.0:1769074794.911187:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475df800, transno 21474841260 00000004:00080000:1.0:1769074794.911189:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef6680, transno 21474841261 00000004:00080000:1.0:1769074794.911192:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dfb80, transno 21474841262 00000004:00080000:1.0:1769074794.911195:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef6a00, transno 21474841263 00000004:00080000:1.0:1769074794.911197:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5475dd880, transno 21474841264 00000100:00100000:1.0:1769074794.911202:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625195136:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1769074794.911418:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074794.911422:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074794.911561:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074794.911572:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625195392:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074794.911603:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1769074794.911621:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 9 done 00000100:00100000:0.0:1769074794.911969:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195392 00000100:00100000:0.0:1769074794.911975:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195392 00000100:00100000:0.0:1769074794.912029:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1182 00000100:00100000:0.0:1769074794.912037:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625195392:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074794.912050:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3905998848 avail 3674476544 00000004:00080000:1.0:1769074794.912082:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 8 done 00000100:00100000:0.0:1769074794.912089:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625195392:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (485us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074794.912096:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1182 00000100:00100000:1.0:1769074794.912120:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625195392:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1769074794.912252:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074795.425726:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074795.425762:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625195520:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074795.425855:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074795.426050:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195520 00000100:00100000:2.0:1769074795.426062:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195520 00000100:00100000:2.0:1769074795.426136:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1183 00000100:00100000:2.0:1769074795.426145:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625195520:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074795.426172:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3900067840 avail 3668545536 00000100:00100000:2.0:1769074795.426220:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625195520:12345-0@lo:13:osp-pre-1-1.0 Request processed in 75us (364us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074795.426228:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1183 00000100:00100000:2.0:1769074795.426304:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625195520:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1769074795.426500:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074795.915456:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074795.915491:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616953728 00000100:00100000:3.0:1769074795.915501:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616953728 00000100:00100000:3.0:1769074795.915556:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4699 00000100:00100000:3.0:1769074795.915566:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+223:63493:x1855007616953728:12345-192.168.204.60@tcp:35:rm.0 00000100:00100000:3.0:1769074795.915791:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63493:x1855007616953728:12345-192.168.204.60@tcp:35:rm.0 Request processed in 225us (353us total) trans 8589944020 rc 0/0 00000100:00100000:3.0:1769074795.915799:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4699 00000100:00080000:3.0:1769074795.915840:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074795.922735:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074795.923560:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616953984 00000100:00100000:3.0:1769074795.923572:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616953984 00000100:00100000:3.0:1769074795.923714:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13377 00000100:00100000:3.0:1769074795.923725:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63493:x1855007616953984:12345-192.168.204.60@tcp:36:rm.0 00010000:00010000:3.0:1769074795.923778:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 5 locks, starting at 0 00010000:00010000:3.0:1769074795.923826:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd546a2 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0xd/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c315 expref: 224 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.923849:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd545d7 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2cf expref: 223 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.923860:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44600/0x7fe2323adfd5461d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2dd expref: 222 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.923872:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867600/0x7fe2323adfd4eef1 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798f59 expref: 221 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.923882:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b665615600) 00010000:00010000:3.0:1769074795.923895:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71000/0x7fe2323adfd4ed2a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198798ec6 expref: 220 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.923903:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074795.923906:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074795.923939:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.923952:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.923960:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.923973:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.923979:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.923984:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.924054:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.924062:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.924068:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000040:00080000:3.0:1769074795.924324:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2456, 64 off210320 00000004:00080000:3.0:1769074795.924334:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1783766526370ns, next at 1784766524337ns, rc = 0 00000040:00080000:3.0:1769074795.924356:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2451, 64 off209616 00000004:00080000:3.0:1769074795.924362:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1783766554927ns, next at 1784766553657ns, rc = 0 00000040:00080000:3.0:1769074795.924373:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2452, 64 off209680 00000040:00080000:3.0:1769074795.924383:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2457, 64 off210384 00000040:00080000:3.0:1769074795.924391:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2458, 64 off210448 00000040:00080000:3.0:1769074795.924400:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2453, 64 off209744 00000040:00080000:3.0:1769074795.924408:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2454, 64 off209808 00000040:00080000:3.0:1769074795.924416:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2455, 64 off209872 00000040:00080000:3.0:1769074795.924424:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2459, 64 off210512 00000040:00080000:3.0:1769074795.924431:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2456, 64 off209936 00000040:00080000:3.0:1769074795.925255:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2460, 64 off210576 00000040:00080000:3.0:1769074795.925281:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2457, 64 off210000 00000040:00080000:3.0:1769074795.925291:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2461, 64 off210640 00000040:00080000:3.0:1769074795.925299:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2462, 64 off210704 00000040:00080000:3.0:1769074795.925307:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2463, 64 off210768 00000040:00080000:3.0:1769074795.925316:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2464, 64 off210832 00000040:00080000:3.0:1769074795.925325:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2458, 64 off210064 00010000:00010000:3.0:1769074795.925392:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925405:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925413:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925421:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925427:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925433:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925829:0:6514:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074795.925856:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925863:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925869:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925875:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b446200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074795.925879:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925886:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925897:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b681690f00) 00010000:00010000:3.0:1769074795.925900:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1769074795.925904:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.925909:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd5474a lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926136:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926143:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926149:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926154:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926159:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b446a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074795.926162:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926171:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926179:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074795.926182:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074795.926185:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926190:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd5473c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000004:00010000:3.0:1769074795.926200:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008766632d reply state 0000000084609100 transno 8589944021 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926209:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926214:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b681690300) 00010000:00010000:3.0:1769074795.926217:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074795.926219:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074795.926304:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:63493:x1855007616953984:12345-192.168.204.60@tcp:36:rm.0 Request processed in 2577us (3575us total) trans 8589944021 rc 0/0 00000100:00100000:3.0:1769074795.926314:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13377 00010000:00010000:3.0:1769074795.926363:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074795.926369:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926375:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074795.926378:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926385:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926393:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fb:0x0].0x0 (ffff89b681690f00) 00010000:00010000:3.0:1769074795.926396:0:6502:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1769074795.926399:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 1/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926404:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54751 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074795.926417:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074795.926418:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074795.926420:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074795.936120:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074795.937502:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074795.938680:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954112 00000100:00100000:2.0:1769074795.938692:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954112 00000100:00100000:2.0:1769074795.938770:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13378 00000100:00100000:2.0:1769074795.938781:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+221:63493:x1855007616954112:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:2.0:1769074795.938822:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074795.938847:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54758 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c35b expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074795.938913:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd5475f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074795.938938:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd5475f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074795.938950:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd5475f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074795.938994:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074795.938999:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074795.939018:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd5475f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074795.939040:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54758 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c35b expref: 223 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074795.939056:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd5475f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c35b expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074795.939064:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074795.939068:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074795.939071:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88600, rc 0) 00000100:00100000:2.0:1769074795.939132:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63493:x1855007616954112:12345-192.168.204.60@tcp:101:rm.0 Request processed in 353us (1638us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074795.939139:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13378 00000100:00100000:3.0:1769074795.941700:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074795.941944:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954240 00000100:00100000:2.0:1769074795.941954:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954240 00000100:00100000:2.0:1769074795.942039:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13379 00000100:00100000:2.0:1769074795.942050:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63493:x1855007616954240:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:2.0:1769074795.942068:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074795.942088:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54766 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c362 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074795.942146:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942159:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942170:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942185:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942192:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942198:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942239:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942246:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942252:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942258:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942274:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074795.942278:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942286:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942298:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074795.942301:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074795.942306:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942312:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5476d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942323:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942328:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942334:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942339:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074795.942342:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942347:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942353:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3a1902b1 (ffff89b64effae00) 00010000:00010000:2.0:1769074795.942356:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074795.942359:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942363:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54774 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3a1902b1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074795.942394:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54766 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c362 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074795.942403:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074795.942405:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074795.942408:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54766 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c362 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074795.942416:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:2.0:1769074795.942490:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63493:x1855007616954240:12345-192.168.204.60@tcp:101:rm.0 Request processed in 443us (800us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074795.942498:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13379 00000100:00100000:2.0:1769074795.946204:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074795.946316:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954368 00000100:00100000:3.0:1769074795.946326:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954368 00000100:00100000:3.0:1769074795.946382:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13380 00000100:00100000:3.0:1769074795.946391:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63493:x1855007616954368:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:3.0:1769074795.946409:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074795.946428:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd5477b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c369 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.946491:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946505:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946515:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946526:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946533:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946537:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946636:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946641:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946645:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946648:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946652:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074795.946655:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946659:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946667:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074795.946669:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074795.946672:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946675:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54782 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946683:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946687:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946690:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946693:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b446a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074795.946694:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946697:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946701:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfbdcc6f6 (ffff89b681690f00) 00010000:00010000:3.0:1769074795.946702:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074795.946703:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946706:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446a00/0x7fe2323adfd54789 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfbdcc6f6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074795.946728:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd5477b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c369 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.946734:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074795.946736:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074795.946739:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd5477b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c369 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074795.946747:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074795.946895:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63493:x1855007616954368:12345-192.168.204.60@tcp:101:rm.0 Request processed in 504us (699us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074795.946902:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13380 00000100:00100000:3.0:1769074795.970887:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074795.970936:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954496 00000100:00100000:1.0:1769074795.970948:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954496 00000100:00100000:1.0:1769074795.971007:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 339 00000100:00100000:1.0:1769074795.971018:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b09050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007616954496:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074795.971097:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b09050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007616954496:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 81us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074795.971105:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 339 00000100:00100000:0.0:1769074795.971209:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074795.971340:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074795.971367:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074795.971397:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954752 00000100:00100000:0.0:1769074795.971402:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954752 00000100:00100000:0.0:1769074795.971439:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954880 00000100:00100000:0.0:1769074795.971442:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954880 00000100:00100000:0.0:1769074795.971463:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1472 00000100:00100000:0.0:1769074795.971471:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616954752:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074795.971508:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616954752:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 38us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074795.971515:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1472 00000100:00100000:0.0:1769074795.971527:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1473 00000100:00100000:0.0:1769074795.971532:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616954880:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074795.971556:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007616954880:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074795.971569:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1473 00000100:00080000:0.0:1769074795.971592:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074795.971596:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074795.971647:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616954624 00000100:00100000:0.0:1769074795.971651:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616954624 00000100:00100000:0.0:1769074795.971687:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2356 00000100:00100000:0.0:1769074795.971694:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2406:x1855007616954624:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074795.971728:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2406:x1855007616954624:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 34us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074795.971734:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2356 00000100:00080000:0.0:1769074795.971747:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:0.0:1769074796.957216:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074797.469703:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625196032:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.469774:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074797.469851:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625196544:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.469876:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074797.469906:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625197056:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.469929:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074797.469955:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1769074797.469962:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625195904:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.469983:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074797.470000:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625196416:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.470018:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074797.470035:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625196928:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.470055:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074797.470088:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196544 00000100:00100000:1.0:1769074797.470095:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196544 00000100:00100000:1.0:1769074797.470145:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195904 00000100:00100000:1.0:1769074797.470149:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195904 00000100:00100000:1.0:1769074797.470176:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196416 00000100:00100000:1.0:1769074797.470179:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196416 00000100:00100000:1.0:1769074797.470199:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2357 00000100:00100000:1.0:1769074797.470206:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625196544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.470262:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625196544:12345-0@lo:400:kworker.0 Request processed in 55us (384us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074797.470269:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2357 00000100:00100000:1.0:1769074797.470300:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2358 00000100:00100000:1.0:1769074797.470306:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625195904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.470333:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625195904:12345-0@lo:400:kworker.0 Request processed in 28us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074797.470339:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2358 00000100:00100000:3.0:1769074797.470450:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625195648:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.470491:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.470516:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625196160:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.470536:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.470553:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625196672:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.470571:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.470580:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625197184:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.470599:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.470610:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1769074797.470615:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625195776:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.470639:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.470656:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625196288:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.470675:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.470683:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625196800:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.470865:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2359 00000100:00100000:0.0:1769074797.470958:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196672 00000100:00100000:0.0:1769074797.470964:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196672 00000100:00100000:0.0:1769074797.470990:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197184 00000100:00100000:0.0:1769074797.470993:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197184 00000100:00100000:0.0:1769074797.471002:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196288 00000100:00100000:0.0:1769074797.471004:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196288 00000100:00100000:1.0:1769074797.472927:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625196416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074797.472990:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195648 00000100:00100000:2.0:1769074797.472997:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195648 00000100:00100000:1.0:1769074797.473027:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625196416:12345-0@lo:400:kworker.0 Request processed in 2152us (3007us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074797.473054:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:2.0:1769074797.473064:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fb9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625195648:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074797.473100:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fb9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625195648:12345-0@lo:400:kworker.0 Request processed in 39us (2607us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074797.473106:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1769074797.473405:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074797.473455:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625195648:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473488:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196032 00000100:00100000:3.0:1769074797.473492:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196032 00000100:00100000:3.0:1769074797.473527:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197056 00000100:00100000:3.0:1769074797.473530:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197056 00000100:00100000:3.0:1769074797.473550:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196928 00000100:00100000:3.0:1769074797.473553:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196928 00000100:00100000:3.0:1769074797.473572:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196160 00000100:00100000:3.0:1769074797.473576:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196160 00000100:00100000:3.0:1769074797.473596:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1474 00000100:00100000:3.0:1769074797.473602:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625196032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473637:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625196032:12345-0@lo:400:kworker.0 Request processed in 35us (3863us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074797.473643:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1474 00000100:00100000:3.0:1769074797.473655:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1475 00000100:00100000:3.0:1769074797.473661:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625197056:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473687:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625197056:12345-0@lo:400:kworker.0 Request processed in 27us (3758us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074797.473692:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1475 00000100:00100000:3.0:1769074797.473702:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1476 00000100:00100000:3.0:1769074797.473708:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625196928:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473736:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625196928:12345-0@lo:400:kworker.0 Request processed in 28us (3680us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074797.473740:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1476 00000100:00100000:3.0:1769074797.473751:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1477 00000100:00100000:3.0:1769074797.473757:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625196160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473786:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625196160:12345-0@lo:400:kworker.0 Request processed in 30us (3251us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074797.473791:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1477 00000100:00100000:3.0:1769074797.473825:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625195776 00000100:00100000:3.0:1769074797.473830:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625195776 00000100:00100000:3.0:1769074797.473853:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625196800 00000100:00100000:3.0:1769074797.473856:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625196800 00000100:00100000:3.0:1769074797.473873:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2662 00000100:00100000:3.0:1769074797.473879:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625195776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473922:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625195776:12345-0@lo:400:kworker.0 Request processed in 43us (3281us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074797.473927:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2662 00000100:00100000:3.0:1769074797.473939:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2663 00000100:00100000:3.0:1769074797.473944:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625196800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.473971:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625196800:12345-0@lo:400:kworker.0 Request processed in 27us (3269us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074797.473976:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2663 00000100:00100000:3.0:1769074797.474024:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625195776:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.474035:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625196800:0@lo:400:kworker.0 00000100:00100000:3.0:1769074797.474045:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625196160:0@lo:400:kworker.0 00000100:00100000:0.0:1769074797.475855:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2360 00000100:00100000:0.0:1769074797.475878:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625196672:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074797.475973:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625196672:12345-0@lo:400:kworker.0 Request processed in 103us (5400us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074797.478122:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2359 00000100:00100000:1.0:1769074797.478177:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2361 00000100:00100000:1.0:1769074797.478190:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625197184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478292:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625197184:12345-0@lo:400:kworker.0 Request processed in 107us (7690us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074797.478300:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2361 00000100:00100000:1.0:1769074797.478316:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2362 00000100:00100000:1.0:1769074797.478324:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625196288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478368:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625196288:12345-0@lo:400:kworker.0 Request processed in 46us (7692us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074797.478374:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2362 00000100:00080000:1.0:1769074797.478405:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074797.478411:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074797.478414:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074797.478416:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074797.478797:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625195904:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478837:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625196416:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478850:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625196928:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478859:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625196032:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478869:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625196544:0@lo:400:kworker.0 00000100:00100000:1.0:1769074797.478879:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625197056:0@lo:400:kworker.0 00000100:00100000:0.0:1769074797.488052:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2360 00000100:00100000:0.0:1769074797.488141:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625196288:0@lo:400:kworker.0 00000100:00100000:0.0:1769074797.488165:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625196672:0@lo:400:kworker.0 00000100:00100000:0.0:1769074797.488180:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625197184:0@lo:400:kworker.0 00010000:00010000:3.0:1769074797.980950:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1769074798.016593:0:36730:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 200c: layout change racing with open: LOVEA changes ========================================================== 04:39:55 (1769074795) 00000100:00100000:1.0:1769074798.292552:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.292594:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955008 00000100:00100000:1.0:1769074798.292605:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955008 00000100:00100000:1.0:1769074798.292667:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 341 00000100:00100000:1.0:1769074798.292676:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fba850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:63659:x1855007616955008:12345-192.168.204.60@tcp:101:mount.lustre.0 00010000:00010000:1.0:1769074798.292693:0:32382:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074798.292711:0:32382:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff89b547866000/0x7fe2323adfd54790 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c370 expref: -99 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074798.292736:0:32382:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff89b547866000/0x7fe2323adfd54790 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c370 expref: 10 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074798.292746:0:32382:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff89b547866000/0x7fe2323adfd54790 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c370 expref: 10 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074798.292755:0:32382:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866000, rc 0) 00000100:00100000:1.0:1769074798.292853:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fba850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63659:x1855007616955008:12345-192.168.204.60@tcp:101:mount.lustre.0 Request processed in 177us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.292860:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 341 00000100:00100000:1.0:1769074798.304192:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.304285:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955136 00000100:00100000:3.0:1769074798.304295:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955136 00000100:00100000:3.0:1769074798.304346:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 342 00000100:00100000:3.0:1769074798.304580:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fb8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63659:x1855007616955136:12345-192.168.204.60@tcp:501:mount.lustre.0 00000100:00100000:3.0:1769074798.304778:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fb8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63659:x1855007616955136:12345-192.168.204.60@tcp:501:mount.lustre.0 Request processed in 423us (596us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.304787:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 342 00000100:00100000:1.0:1769074798.317996:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.318070:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955264 00000100:00100000:3.0:1769074798.318080:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955264 00000100:00100000:3.0:1769074798.318133:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 343 00000100:00100000:3.0:1769074798.318152:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fb9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63659:x1855007616955264:12345-192.168.204.60@tcp:503:mount.lustre.0 00000100:00100000:3.0:1769074798.318294:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fb9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63659:x1855007616955264:12345-192.168.204.60@tcp:503:mount.lustre.0 Request processed in 153us (306us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.318302:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 343 00000100:00100000:1.0:1769074798.332109:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.332640:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955392 00000100:00100000:2.0:1769074798.332651:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955392 00000100:00100000:2.0:1769074798.332714:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 344 00000100:00100000:2.0:1769074798.332724:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63665:x1855007616955392:12345-192.168.204.60@tcp:502:llog_process_th.0 00000100:00100000:2.0:1769074798.332949:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63665:x1855007616955392:12345-192.168.204.60@tcp:502:llog_process_th.0 Request processed in 226us (849us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.332957:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 344 00000100:00100000:3.0:1769074798.352673:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.352911:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955520 00000100:00100000:1.0:1769074798.352922:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955520 00000100:00100000:1.0:1769074798.352979:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 345 00000100:00100000:1.0:1769074798.352989:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63665:x1855007616955520:12345-192.168.204.60@tcp:502:llog_process_th.0 00000100:00100000:1.0:1769074798.353140:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63665:x1855007616955520:12345-192.168.204.60@tcp:502:llog_process_th.0 Request processed in 153us (487us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.353149:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 345 00000100:00100000:1.0:1769074798.376728:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.376888:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955648 00000100:00100000:3.0:1769074798.376899:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955648 00000100:00100000:3.0:1769074798.377085:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 346 00000100:00100000:3.0:1769074798.377096:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63665:x1855007616955648:12345-192.168.204.60@tcp:502:llog_process_th.0 00000100:00100000:3.0:1769074798.377265:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63665:x1855007616955648:12345-192.168.204.60@tcp:502:llog_process_th.0 Request processed in 169us (545us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.377274:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 346 00000100:00100000:1.0:1769074798.382507:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.383074:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955776 00000100:00100000:2.0:1769074798.383085:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955776 00000100:00100000:2.0:1769074798.383149:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 347 00000100:00100000:2.0:1769074798.383159:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+10:63659:x1855007616955776:12345-192.168.204.60@tcp:101:mount.lustre.0 00010000:00010000:2.0:1769074798.383175:0:32382:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074798.383192:0:32382:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff89b666b89c00/0x7fe2323adfd54797 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3a1 expref: -99 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.383223:0:32382:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff89b666b89c00/0x7fe2323adfd54797 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3a1 expref: 11 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.383235:0:32382:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff89b666b89c00/0x7fe2323adfd54797 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3a1 expref: 11 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.383244:0:32382:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89c00, rc 0) 00000100:00100000:2.0:1769074798.383311:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+11:63659:x1855007616955776:12345-192.168.204.60@tcp:101:mount.lustre.0 Request processed in 153us (811us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.383321:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 347 00000100:00100000:0.0:1769074798.396696:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.396912:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616955904 00000100:00100000:3.0:1769074798.396922:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616955904 00000100:00100000:3.0:1769074798.396982:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 348 00000100:00100000:3.0:1769074798.396992:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+11:63659:x1855007616955904:12345-192.168.204.60@tcp:256:mount.lustre.0 20000000:01000000:3.0:1769074798.397021:0:32381:0:(mgs_nids.c:911:mgs_get_ir_logs()) Reading IR log lustre-cliir bufsize 1048576. 20000000:01000000:3.0:1769074798.397044:0:32381:0:(mgs_nids.c:305:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4052/1/256/255. 20000000:01000000:3.0:1769074798.397047:0:32381:0:(mgs_nids.c:316:mgs_nidtbl_read()) Read IR logs lustre return with 44, version 14 00000100:00100000:3.0:1769074798.408402:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+11:63659:x1855007616955904:12345-192.168.204.60@tcp:256:mount.lustre.0 Request processed in 11398us (11704us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.408417:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 348 00000100:00100000:1.0:1769074798.424930:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.425088:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956032 00000100:00100000:3.0:1769074798.425099:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956032 00000100:00100000:3.0:1769074798.425170:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 349 00000100:00100000:3.0:1769074798.425181:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+11:63659:x1855007616956032:12345-192.168.204.60@tcp:101:mount.lustre.0 00010000:00010000:3.0:1769074798.425200:0:32381:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074798.425218:0:32381:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff89b67b446a00/0x7fe2323adfd5479e lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3a8 expref: -99 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.425258:0:32381:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff89b67b446a00/0x7fe2323adfd5479e lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3a8 expref: 12 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.425268:0:32381:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff89b67b446a00/0x7fe2323adfd5479e lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3a8 expref: 12 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.425276:0:32381:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67b446a00, rc 0) 00000100:00100000:3.0:1769074798.425332:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63659:x1855007616956032:12345-192.168.204.60@tcp:101:mount.lustre.0 Request processed in 153us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.425339:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 349 00000100:00100000:0.0:1769074798.430846:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.431427:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956160 00000100:00100000:3.0:1769074798.431439:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956160 00000100:00100000:3.0:1769074798.431496:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 350 00000100:00100000:3.0:1769074798.431505:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63659:x1855007616956160:12345-192.168.204.60@tcp:501:mount.lustre.0 00000100:00100000:3.0:1769074798.431639:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63659:x1855007616956160:12345-192.168.204.60@tcp:501:mount.lustre.0 Request processed in 135us (847us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.431646:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 350 00000100:00100000:1.0:1769074798.434262:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.434435:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956288 00000100:00100000:2.0:1769074798.434446:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956288 00000100:00100000:2.0:1769074798.434501:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 351 00000100:00100000:2.0:1769074798.434511:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63659:x1855007616956288:12345-192.168.204.60@tcp:503:mount.lustre.0 00000100:00100000:2.0:1769074798.434706:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63659:x1855007616956288:12345-192.168.204.60@tcp:503:mount.lustre.0 Request processed in 196us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.434714:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 351 00000100:00100000:1.0:1769074798.438479:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.438730:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956416 00000100:00100000:3.0:1769074798.438741:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956416 00000100:00100000:3.0:1769074798.438797:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 352 00000100:00100000:3.0:1769074798.438841:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681596450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63670:x1855007616956416:12345-192.168.204.60@tcp:502:llog_process_th.0 00000100:00100000:3.0:1769074798.439003:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681596450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63670:x1855007616956416:12345-192.168.204.60@tcp:502:llog_process_th.0 Request processed in 197us (533us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.439015:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 352 00000100:00100000:0.0:1769074798.451420:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.451590:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956544 00000100:00100000:2.0:1769074798.451601:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956544 00000100:00100000:2.0:1769074798.451659:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 353 00000100:00100000:2.0:1769074798.451669:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681595050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63670:x1855007616956544:12345-192.168.204.60@tcp:502:llog_process_th.0 00000100:00100000:2.0:1769074798.451889:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681595050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:63670:x1855007616956544:12345-192.168.204.60@tcp:502:llog_process_th.0 Request processed in 220us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.451897:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 353 00000100:00100000:1.0:1769074798.469988:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.470028:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.470065:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.470098:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.470125:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.470155:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956672 00000100:00100000:1.0:1769074798.470164:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956672 00000100:00100000:1.0:1769074798.470197:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956928 00000100:00100000:1.0:1769074798.470200:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956928 00000100:00100000:1.0:1769074798.470209:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957056 00000100:00100000:1.0:1769074798.470211:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957056 00000100:00100000:1.0:1769074798.470241:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957312 00000100:00100000:1.0:1769074798.470243:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957312 00000100:00100000:1.0:1769074798.470262:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2363 00000100:00100000:1.0:1769074798.470271:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:2407:x1855007616957056:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074798.470342:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:2407:x1855007616957056:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 72us (276us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.470360:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2363 00000100:00100000:1.0:1769074798.470378:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2364 00000100:00100000:1.0:1769074798.470384:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616957312:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074798.470406:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2407:x1855007616957312:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 22us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.470412:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2364 00000100:00100000:1.0:1769074798.470422:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13381 00000100:00100000:1.0:1769074798.470426:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2405:x1855007616956672:12345-192.168.204.60@tcp:38:mount.lustre.0 00010000:00080000:1.0:1769074798.470447:0:8855:0:(ldlm_lib.c:1428:target_handle_connect()) lustre-MDT0000: connection from 2274e3cf-6162-4197-bb1e-d97abc9b4bb9@192.168.204.60@tcp t0 exp 0000000000000000 cur 1786 last 0 00000020:00000080:1.0:1769074798.470478:0:8855:0:(genops.c:1402:class_connect()) connect: client 2274e3cf-6162-4197-bb1e-d97abc9b4bb9, cookie 0x7fe2323adfd547a5 00000004:00100000:1.0:1769074798.470501:0:8855:0:(mdt_handler.c:7289:mdt_connect_internal()) lustre-MDT0000: cli supports cksum type f7, return f7 00000020:01000000:1.0:1769074798.470567:0:8855:0:(lprocfs_status_server.c:658:lprocfs_exp_setup()) using hash ffff89b645fbc800 00000100:00080000:1.0:1769074798.470606:0:8855:0:(import.c:65:import_set_state_nolock()) ffff89b643aae800 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1769074798.470635:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2405:x1855007616956672:12345-192.168.204.60@tcp:38:mount.lustre.0 Request processed in 207us (657us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.470642:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13381 00000100:00100000:1.0:1769074798.470655:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13382 00000100:00100000:1.0:1769074798.470658:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2405:x1855007616956928:12345-192.168.204.60@tcp:38:mount.lustre.0 00010000:00080000:1.0:1769074798.470672:0:8855:0:(ldlm_lib.c:1428:target_handle_connect()) lustre-MDT0001: connection from 2274e3cf-6162-4197-bb1e-d97abc9b4bb9@192.168.204.60@tcp t0 exp 0000000000000000 cur 1786 last 0 00000020:00000080:1.0:1769074798.470697:0:8855:0:(genops.c:1402:class_connect()) connect: client 2274e3cf-6162-4197-bb1e-d97abc9b4bb9, cookie 0x7fe2323adfd547ac 00000004:00100000:1.0:1769074798.470709:0:8855:0:(mdt_handler.c:7289:mdt_connect_internal()) lustre-MDT0001: cli supports cksum type f7, return f7 00000020:01000000:1.0:1769074798.470743:0:8855:0:(lprocfs_status_server.c:658:lprocfs_exp_setup()) using hash ffff89b6760bc500 00000100:00080000:1.0:1769074798.470761:0:8855:0:(import.c:65:import_set_state_nolock()) ffff89b54361a000 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1769074798.470787:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2405:x1855007616956928:12345-192.168.204.60@tcp:38:mount.lustre.0 Request processed in 127us (759us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.470794:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13382 00000100:00100000:1.0:1769074798.470854:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957568 00000100:00100000:1.0:1769074798.470858:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957568 00000100:00100000:1.0:1769074798.470903:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1478 00000100:00100000:1.0:1769074798.470910:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616957568:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074798.470937:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2407:x1855007616957568:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 28us (811us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.470943:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1478 00000100:00080000:1.0:1769074798.470964:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074798.470968:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074798.470970:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074798.480895:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.481040:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957696 00000100:00100000:0.0:1769074798.481052:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957696 00000100:00100000:0.0:1769074798.481130:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13383 00000100:00100000:0.0:1769074798.481141:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616957696:12345-192.168.204.60@tcp:41:mount.lustre.0 00000004:00000024:0.0:1769074798.481184:0:6513:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00080000:1.0:1769074798.481533:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074798.481597:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616957696:12345-192.168.204.60@tcp:41:mount.lustre.0 Request processed in 458us (722us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074798.481607:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13383 00000100:00100000:0.0:1769074798.486888:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.486932:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.486966:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.487054:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616956800 00000100:00100000:2.0:1769074798.487064:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616956800 00000100:00100000:2.0:1769074798.487120:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957184 00000100:00100000:2.0:1769074798.487123:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957184 00000100:00100000:2.0:1769074798.487159:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13384 00000100:00100000:2.0:1769074798.487168:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:0:x1855007616956800:12345-192.168.204.60@tcp:41:mount.lustre.0 00000004:00000024:2.0:1769074798.487199:0:6513:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00100000:2.0:1769074798.487287:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:0:x1855007616956800:12345-192.168.204.60@tcp:41:mount.lustre.0 Request processed in 119us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.487295:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13384 00000100:00100000:2.0:1769074798.487313:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13385 00000100:00100000:2.0:1769074798.487320:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2406:x1855007616957184:12345-192.168.204.60@tcp:41:mount.lustre.0 00000004:00000024:2.0:1769074798.487338:0:6513:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:2.0:1769074798.487377:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2406:x1855007616957184:12345-192.168.204.60@tcp:41:mount.lustre.0 Request processed in 57us (444us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.487384:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13385 00000100:00080000:2.0:1769074798.487404:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074798.488014:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957440 00000100:00100000:1.0:1769074798.488022:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957440 00000100:00100000:1.0:1769074798.488087:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1479 00000100:00100000:1.0:1769074798.488095:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616957440:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074798.488139:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007616957440:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 45us (1172us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074798.488147:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1479 00000100:00080000:2.0:1769074798.488767:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074798.491565:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.491670:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957824 00000100:00100000:3.0:1769074798.491681:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957824 00000100:00100000:3.0:1769074798.491742:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13386 00000100:00100000:3.0:1769074798.491752:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616957824:12345-192.168.204.60@tcp:40:mount.lustre.0 00000100:00100000:3.0:1769074798.491884:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616957824:12345-192.168.204.60@tcp:40:mount.lustre.0 Request processed in 126us (322us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.491893:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13386 00000100:00100000:1.0:1769074798.497027:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.497233:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616957952 00000100:00100000:0.0:1769074798.497243:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616957952 00000100:00100000:0.0:1769074798.497290:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14407 00000100:00100000:0.0:1769074798.497299:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:0+-99:2405:x1855007616957952:12345-192.168.204.60@tcp:8:mount.lustre.0 00010000:00080000:0.0:1769074798.497330:0:28954:0:(ldlm_lib.c:1428:target_handle_connect()) lustre-OST0000: connection from 2274e3cf-6162-4197-bb1e-d97abc9b4bb9@192.168.204.60@tcp t0 exp 0000000000000000 cur 1786 last 0 00000020:00000080:0.0:1769074798.497523:0:28954:0:(genops.c:1402:class_connect()) connect: client 2274e3cf-6162-4197-bb1e-d97abc9b4bb9, cookie 0x7fe2323adfd547b3 00002000:00100000:0.0:1769074798.497536:0:28954:0:(ofd_obd.c:124:ofd_parse_connect_data()) lustre-OST0000: cli 2274e3cf-6162-4197-bb1e-d97abc9b4bb9/ffff89b649256800 ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 2113204 ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1769074798.497553:0:28954:0:(ofd_obd.c:222:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00000020:01000000:0.0:1769074798.497619:0:28954:0:(lprocfs_status_server.c:658:lprocfs_exp_setup()) using hash ffff89b6760bc700 00002000:00080000:0.0:1769074798.497629:0:28954:0:(ofd_obd.c:365:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:0.0:1769074798.497658:0:28954:0:(import.c:65:import_set_state_nolock()) ffff89b649251800 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1769074798.498046:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2405:x1855007616957952:12345-192.168.204.60@tcp:8:mount.lustre.0 Request processed in 745us (1026us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074798.498055:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14407 00000100:00100000:1.0:1769074798.506598:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074798.506651:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.506854:0:6524:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958208 00000100:00100000:3.0:1769074798.506865:0:6524:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958208 00000100:00100000:3.0:1769074798.506935:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 26 00000100:00100000:3.0:1769074798.506945:0:6524:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616958208:12345-192.168.204.60@tcp:900:mount.lustre.0 00000100:00100000:3.0:1769074798.507680:0:6524:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616958208:12345-192.168.204.60@tcp:900:mount.lustre.0 Request processed in 86us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.507693:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 26 00000100:00100000:2.0:1769074798.507800:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958080 00000100:00100000:2.0:1769074798.507838:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958080 00000100:00100000:2.0:1769074798.507873:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14408 00000100:00100000:2.0:1769074798.507887:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0+-99:2405:x1855007616958080:12345-192.168.204.60@tcp:8:mount.lustre.0 00010000:00080000:2.0:1769074798.507916:0:8399:0:(ldlm_lib.c:1428:target_handle_connect()) lustre-OST0001: connection from 2274e3cf-6162-4197-bb1e-d97abc9b4bb9@192.168.204.60@tcp t0 exp 0000000000000000 cur 1786 last 0 00000020:00000080:2.0:1769074798.507945:0:8399:0:(genops.c:1402:class_connect()) connect: client 2274e3cf-6162-4197-bb1e-d97abc9b4bb9, cookie 0x7fe2323adfd547ba 00002000:00100000:2.0:1769074798.507957:0:8399:0:(ofd_obd.c:124:ofd_parse_connect_data()) lustre-OST0001: cli 2274e3cf-6162-4197-bb1e-d97abc9b4bb9/ffff89b66e785000 ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 2113204 ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1769074798.507973:0:8399:0:(ofd_obd.c:222:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00000020:01000000:2.0:1769074798.508188:0:8399:0:(lprocfs_status_server.c:658:lprocfs_exp_setup()) using hash ffff89b66e700200 00002000:00080000:2.0:1769074798.508199:0:8399:0:(ofd_obd.c:365:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1769074798.508235:0:8399:0:(import.c:65:import_set_state_nolock()) ffff89b66e781000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1769074798.508322:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2405:x1855007616958080:12345-192.168.204.60@tcp:8:mount.lustre.0 Request processed in 434us (1670us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.508329:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14408 00000100:00100000:1.0:1769074798.515287:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.515559:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958336 00000100:00100000:3.0:1769074798.515571:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958336 00000100:00100000:3.0:1769074798.515633:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4700 00000100:00100000:3.0:1769074798.515642:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616958336:12345-192.168.204.60@tcp:33:mount.lustre.0 00000004:00200000:3.0:1769074798.515729:0:6516:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000100:00100000:3.0:1769074798.515847:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63659:x1855007616958336:12345-192.168.204.60@tcp:33:mount.lustre.0 Request processed in 205us (570us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.515855:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4700 00000100:00100000:1.0:1769074798.767383:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.767635:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958464 00000100:00100000:2.0:1769074798.767645:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958464 00000100:00100000:2.0:1769074798.767709:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13387 00000100:00100000:2.0:1769074798.767719:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63691:x1855007616958464:12345-192.168.204.60@tcp:41:lfs.0 00000004:00000024:2.0:1769074798.767749:0:6512:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00100000:2.0:1769074798.767868:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63691:x1855007616958464:12345-192.168.204.60@tcp:41:lfs.0 Request processed in 150us (496us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.767886:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13387 00000100:00100000:1.0:1769074798.772600:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.772796:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958592 00000100:00100000:3.0:1769074798.772830:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958592 00000100:00100000:3.0:1769074798.772899:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13388 00000100:00100000:3.0:1769074798.772910:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:63691:x1855007616958592:12345-192.168.204.60@tcp:41:lfs.0 00000004:00000024:3.0:1769074798.772942:0:10283:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074798.773017:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:63691:x1855007616958592:12345-192.168.204.60@tcp:41:lfs.0 Request processed in 109us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.773025:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13388 00000100:00100000:1.0:1769074798.776851:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.777055:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958720 00000100:00100000:2.0:1769074798.777068:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958720 00000100:00100000:2.0:1769074798.777127:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1184 00000100:00100000:2.0:1769074798.777150:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:63691:x1855007616958720:12345-192.168.204.60@tcp:13:lfs.0 00002000:00000024:2.0:1769074798.777176:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3905998848 avail 3674476544 00000100:00100000:2.0:1769074798.777252:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:63691:x1855007616958720:12345-192.168.204.60@tcp:13:lfs.0 Request processed in 116us (457us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.777259:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1184 00000100:00100000:1.0:1769074798.780737:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.780938:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958848 00000100:00100000:0.0:1769074798.780948:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958848 00000100:00100000:0.0:1769074798.781008:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1185 00000100:00100000:0.0:1769074798.781017:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:63691:x1855007616958848:12345-192.168.204.60@tcp:13:lfs.0 00002000:00000024:0.0:1769074798.781043:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3900067840 avail 3668545536 00000100:00100000:0.0:1769074798.781115:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:63691:x1855007616958848:12345-192.168.204.60@tcp:13:lfs.0 Request processed in 99us (389us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074798.781121:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1185 00000100:00100000:1.0:1769074798.787383:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.787480:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616958976 00000100:00100000:2.0:1769074798.787491:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616958976 00000100:00100000:2.0:1769074798.787550:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13389 00000100:00100000:2.0:1769074798.787559:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616958976:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074798.787580:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074798.787601:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd547c1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3d9 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.787672:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074798.787688:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074798.787702:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074798.787752:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074798.787890:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074798.787913:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074798.787936:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd547c1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.787953:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 7 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.787962:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074798.787965:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074798.787968:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88400, rc 0) 00000100:00100000:2.0:1769074798.788030:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:63691:x1855007616958976:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 472us (657us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.788037:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13389 00000100:00100000:1.0:1769074798.791039:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.792482:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616959104 00000100:00100000:3.0:1769074798.792496:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616959104 00000100:00100000:3.0:1769074798.792779:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13390 00000100:00100000:3.0:1769074798.792790:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616959104:12345-192.168.204.60@tcp:41:lfs.0 00000004:00000024:3.0:1769074798.792840:0:10283:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074798.792939:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616959104:12345-192.168.204.60@tcp:41:lfs.0 Request processed in 152us (1909us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074798.792947:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13390 00000100:00100000:1.0:1769074798.797003:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.797665:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616959232 00000100:00100000:2.0:1769074798.797676:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616959232 00000100:00100000:2.0:1769074798.797760:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1186 00000100:00100000:2.0:1769074798.797770:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616959232:12345-192.168.204.60@tcp:13:lfs.0 00002000:00000024:2.0:1769074798.797796:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3905998848 avail 3674476544 00000100:00100000:2.0:1769074798.797899:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616959232:12345-192.168.204.60@tcp:13:lfs.0 Request processed in 131us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.797907:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1186 00000100:00080000:3.0:1769074798.798072:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074798.808018:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.815933:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616959360 00000100:00100000:0.0:1769074798.815951:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616959360 00000100:00100000:0.0:1769074798.816098:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1187 00000100:00100000:0.0:1769074798.816108:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616959360:12345-192.168.204.60@tcp:13:lfs.0 00002000:00000024:0.0:1769074798.816154:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3900067840 avail 3668545536 00000100:00100000:0.0:1769074798.816240:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63691:x1855007616959360:12345-192.168.204.60@tcp:13:lfs.0 Request processed in 134us (8231us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074798.816248:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1187 00000100:00080000:3.0:1769074798.830911:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074798.839297:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.839492:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616959616 00000100:00100000:3.0:1769074798.839504:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616959616 00000100:00100000:3.0:1769074798.839568:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13391 00000100:00100000:3.0:1769074798.839577:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+222:63692:x1855007616959616:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074798.839597:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074798.839600:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074798.839625:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd5475f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c35b expref: 222 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.839646:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074798.839652:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074798.839666:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd547cf lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3e0 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.839761:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.839774:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 6 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.839781:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.839997:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 6 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.840011:0:10283:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 6 pid: 6512 timeout: 1886 lvb_type: 0 00000100:00100000:3.0:1769074798.840024:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625197312:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074798.840109:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074798.848733:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074798.848877:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625197312:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074798.848905:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074798.849079:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616959744 00000100:00100000:2.0:1769074798.849086:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616959744 00010000:00010000:2.0:1769074798.849125:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 7 pid: 6512 timeout: 1886 lvb_type: 0 00000100:00100000:2.0:1769074798.849162:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7177 00000100:00100000:2.0:1769074798.849171:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2408:x1855007616959744:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074798.849186:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074798.849192:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 7 pid: 6512 timeout: 1886 lvb_type: 0 00010000:00010000:2.0:1769074798.849205:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 7 pid: 6512 timeout: 1886 lvb_type: 0 00010000:00010000:2.0:1769074798.849234:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547c8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3d9 expref: 7 pid: 6512 timeout: 1886 lvb_type: 0 00010000:00010000:2.0:1769074798.849249:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074798.849254:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074798.849262:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074798.849269:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074798.849292:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074798.849338:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616959744:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 168us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.849353:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7177 00010000:00010000:3.0:1769074798.849578:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.849585:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.849611:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.849620:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.849626:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00080000:3.0:1769074798.849967:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c1:0x0], index 0: 0 00010000:00010000:3.0:1769074798.850106:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd547e4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850115:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd547e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850122:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd547e4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850156:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850162:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850168:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850174:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850180:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fa3c800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074798.850183:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850190:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850199:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074798.850202:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074798.850206:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850211:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c800/0x7fe2323adfd547d6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000004:00010000:3.0:1769074798.850221:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000006c3b8a6b reply state 000000004f642d2e transno 8589944022 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850231:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074798.850236:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b681690f00) 00010000:00010000:3.0:1769074798.850238:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074798.850241:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1769074798.850272:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446200/0x7fe2323adfd547cf lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3e0 expref: 224 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.850285:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd547e4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3e0 expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074798.850292:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b681690500) 00010000:00010000:3.0:1769074798.850294:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074798.850297:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074798.850333:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63692:x1855007616959616:12345-192.168.204.60@tcp:101:dd.0 Request processed in 10758us (11047us total) trans 8589944022 rc 0/0 00000100:00100000:3.0:1769074798.850356:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13391 00000100:00100000:1.0:1769074798.874031:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074798.877517:0:8399:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616959872 00000100:00100000:2.0:1769074798.877533:0:8399:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616959872 00000100:00100000:2.0:1769074798.877676:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14409 00000100:00100000:2.0:1769074798.877689:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:63692:x1855007616959872:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074798.877707:0:8399:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074798.877728:0:8399:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3e7 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074798.877751:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c1:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1769074798.877770:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074798.877772:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074798.877781:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3e7 expref: 21 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.877798:0:8399:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3e7 expref: 21 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074798.877837:0:8399:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88400, rc 0) 00000100:00100000:2.0:1769074798.877924:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:63692:x1855007616959872:12345-192.168.204.60@tcp:101:dd.0 Request processed in 238us (3904us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074798.877932:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14409 00000100:00100000:1.0:1769074798.963278:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074798.968675:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616960128 00000100:00100000:0.0:1769074798.968690:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616960128 00000100:00100000:0.0:1769074798.968763:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4701 00000100:00100000:0.0:1769074798.968773:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+224:63692:x1855007616960128:12345-192.168.204.60@tcp:35:dd.0 00000100:00100000:0.0:1769074798.969095:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63692:x1855007616960128:12345-192.168.204.60@tcp:35:dd.0 Request processed in 322us (5826us total) trans 8589944023 rc 0/0 00000100:00100000:0.0:1769074798.969103:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4701 00010000:00010000:2.0:1769074799.005222:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074799.013736:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074799.014629:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616960256 00000100:00100000:0.0:1769074799.014641:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616960256 00000100:00100000:1.0:1769074799.021291:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13392 00000100:00100000:1.0:1769074799.021306:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+225:63693:x1855007616960256:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074799.021337:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074799.021359:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd547f9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3ee expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074799.021427:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd54800 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074799.021442:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd54800 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074799.021453:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd54800 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074799.021498:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:1.0:1769074799.021502:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074799.021521:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd54800 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074799.021545:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866800/0x7fe2323adfd547f9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3ee expref: 227 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074799.021561:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd54800 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3ee expref: 226 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074799.021570:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074799.021573:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1769074799.021577:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867400, rc 0) 00000100:00100000:1.0:1769074799.021652:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:63693:x1855007616960256:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 349us (7926us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074799.021670:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13392 00000100:00100000:1.0:1769074799.039999:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074799.040262:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616960512 00000100:00100000:2.0:1769074799.040274:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616960512 00000100:00100000:2.0:1769074799.040333:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13393 00000100:00100000:2.0:1769074799.040343:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+226:63693:x1855007616960512:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074799.040363:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074799.040385:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54807 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3f5 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.040496:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5480e lrc: 3/1,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074799.040510:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5480e lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074799.040522:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5480e lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074799.040533:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00010000:00010000:2.0:1769074799.040640:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54807 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3f5 expref: 229 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.040669:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5480e lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3f5 expref: 228 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.040679:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b681690500) 00010000:00010000:2.0:1769074799.040681:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1769074799.040684:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:2.0:1769074799.040746:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:63693:x1855007616960512:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 404us (757us total) trans 8589944024 rc 0/0 00000100:00100000:2.0:1769074799.040753:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13393 00000100:00100000:3.0:1769074799.077640:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074799.077705:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074799.078213:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616960768 00000100:00100000:0.0:1769074799.078225:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616960768 00000100:00100000:0.0:1769074799.078323:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961024 00000100:00100000:0.0:1769074799.078328:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961024 00000100:00100000:3.0:1769074799.078632:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4702 00000100:00100000:3.0:1769074799.078645:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+228:39339:x1855007616960768:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:1.0:1769074799.078781:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4703 00000100:00100000:1.0:1769074799.078792:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:60112:x1855007616961024:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:2.0:1769074799.080894:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:39339:x1855007616960768:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 2240us (3252us total) trans 8589944025 rc 0/0 00000100:00100000:2.0:1769074799.080908:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4702 00000100:00100000:1.0:1769074799.089010:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:60112:x1855007616961024:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 10204us (11287us total) trans 4294968348 rc 0/0 00000100:00100000:1.0:1769074799.089027:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4703 00000100:00100000:3.0:1769074799.093734:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074799.094589:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961280 00000100:00100000:0.0:1769074799.094601:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961280 00000100:00100000:0.0:1769074799.094669:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4704 00000100:00100000:0.0:1769074799.094680:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+229:39339:x1855007616961280:12345-192.168.204.60@tcp:35:ldlm_bl.0 00000100:00100000:0.0:1769074799.095065:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:39339:x1855007616961280:12345-192.168.204.60@tcp:35:ldlm_bl.0 Request processed in 386us (1342us total) trans 8589944026 rc 0/0 00000100:00100000:0.0:1769074799.095074:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4704 00000100:00100000:2.0:1769074799.098856:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074799.099030:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961408 00000100:00100000:1.0:1769074799.099041:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961408 00000100:00100000:1.0:1769074799.099125:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1034 00000100:00100000:1.0:1769074799.099137:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:60112:x1855007616961408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074799.099225:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 11 locks, starting at 0 00010000:00010000:0.0:1769074799.099282:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd4875d lrc: 0/0,0 mode: --/CR res: [0x240000404:0x48:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795592 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099304:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x48:0x0].0x0 (ffff89b665612600) 00010000:00010000:0.0:1769074799.099315:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd486fb lrc: 0/0,0 mode: --/CR res: [0x240000404:0x49:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795576 expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099325:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x49:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074799.099350:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b5495c9800/0x7fe2323adfd485ea lrc: 0/0,0 mode: --/PR res: [0x240000404:0x48:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795514 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099361:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x48:0x0].0x0 (ffff89b665612600) 00010000:00010000:0.0:1769074799.099391:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b64eff2400/0x7fe2323adfd384a3 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f433 expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099411:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x47:0x0].0x0 (ffff89b6519abc00) 00010000:00010000:0.0:1769074799.099436:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b5481b6a00/0x7fe2323adfd1bde9 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860ed expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099445:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x5:0x0].0x0 (ffff89b54362b300) 00010000:00010000:0.0:1769074799.099465:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b5481b6800/0x7fe2323adfd1ba54 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785fdc expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099476:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641650e00/0x7fe2323adfd1ba38 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785fd5 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099486:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b5481b6400/0x7fe2323adfd1b871 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785f2d expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099504:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x44:0x0].0x0 (ffff89b643dac500) 00010000:00010000:0.0:1769074799.099515:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641650c00/0x7fe2323adfd1b71a lrc: 0/0,0 mode: --/PR res: [0x240000404:0x43:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785eb6 expref: 9 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099536:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x43:0x0].0x0 (ffff89b665611200) 00010000:00010000:0.0:1769074799.099547:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641651600/0x7fe2323adfd1b24a lrc: 0/0,0 mode: --/PR res: [0x240000404:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785d20 expref: 8 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099556:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x3d:0x0].0x0 (ffff89b665611f00) 00010000:00010000:0.0:1769074799.099579:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b5491fb600/0x7fe2323adfd1adb2 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x3c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785b9f expref: 7 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.099587:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x3c:0x0].0x0 (ffff89b665611900) 00010000:00010000:0.0:1769074799.099593:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074799.099663:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:60112:x1855007616961408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 529us (862us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074799.099670:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1034 00000100:00100000:3.0:1769074799.109653:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074799.110010:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961536 00000100:00100000:2.0:1769074799.110022:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961536 00000100:00100000:2.0:1769074799.110319:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1035 00000100:00100000:2.0:1769074799.110331:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e666850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+230:39339:x1855007616961536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074799.110356:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 215 locks, starting at 0 00010000:00010000:2.0:1769074799.110383:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5480e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3f5 expref: 230 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110405:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b681690500) 00010000:00010000:2.0:1769074799.110416:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867400/0x7fe2323adfd54800 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3ee expref: 229 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110425:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074799.110432:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd547e4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3e0 expref: 228 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110439:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b681690500) 00010000:00010000:2.0:1769074799.110451:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd48cc0 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1f5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879571a expref: 227 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110463:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f0e600/0x7fe2323adfd48b8c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1f5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987956a3 expref: 226 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110471:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1f5:0x0].0x0 (ffff89b665617e00) 00010000:00010000:2.0:1769074799.110482:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5495c8600/0x7fe2323adfd485d5 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879550d expref: 225 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110491:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x48:0x0].0x0 (ffff89b665612f00) 00010000:00010000:2.0:1769074799.110502:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd48606 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795522 expref: 224 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.110512:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074799.114593:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925a00/0x7fe2323adfd38480 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1ab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f42c expref: 223 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114626:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1ab:0x0].0x0 (ffff89b665617a00) 00010000:00010000:2.0:1769074799.114646:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4400/0x7fe2323adfd383df lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1a8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f3f4 expref: 222 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114660:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4600/0x7fe2323adfd383a0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1a8:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f3d8 expref: 221 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114669:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1a8:0x0].0x0 (ffff89b665618e00) 00010000:00010000:2.0:1769074799.114681:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053a00/0x7fe2323adfd1be3d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786109 expref: 220 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114690:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x27:0x0].0x0 (ffff89b6543ea500) 00010000:00010000:2.0:1769074799.114698:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925000/0x7fe2323adfd3815b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1a4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f345 expref: 219 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114707:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1a4:0x0].0x0 (ffff89b654294b00) 00010000:00010000:2.0:1769074799.114717:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46a00/0x7fe2323adfd380ba lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1a0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f314 expref: 218 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114729:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1e00/0x7fe2323adfd37f9b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1a0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f2b9 expref: 217 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114737:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1a0:0x0].0x0 (ffff89b665617c00) 00010000:00010000:2.0:1769074799.114746:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1800/0x7fe2323adfd37c5a lrc: 0/0,0 mode: --/CR res: [0x200000405:0x19f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f1af expref: 216 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114754:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x19f:0x0].0x0 (ffff89b6542eb700) 00010000:00010000:2.0:1769074799.114764:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd37b3b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x19c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f13f expref: 215 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114776:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925200/0x7fe2323adfd37a31 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x19c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878f0f9 expref: 214 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114784:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x19c:0x0].0x0 (ffff89b6472cbf00) 00010000:00010000:2.0:1769074799.114793:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437a00/0x7fe2323adfd376fe lrc: 0/0,0 mode: --/CR res: [0x200000405:0x19a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878efe8 expref: 213 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114801:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x19a:0x0].0x0 (ffff89b665615900) 00010000:00010000:2.0:1769074799.114842:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4200/0x7fe2323adfd371ef lrc: 0/0,0 mode: --/CR res: [0x200000405:0x194:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ee59 expref: 212 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.114855:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3cc00/0x7fe2323adfd3710f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x194:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ee13 expref: 211 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.115274:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x194:0x0].0x0 (ffff89b665615100) 00010000:00010000:2.0:1769074799.115297:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436e00/0x7fe2323adfd36e45 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x191:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ed2c expref: 210 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.115311:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436200/0x7fe2323adfd36d11 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x191:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ecc3 expref: 209 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.115319:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x191:0x0].0x0 (ffff89b666b4a100) 00010000:00010000:2.0:1769074799.115330:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96600/0x7fe2323adfd36a32 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x190:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ebd5 expref: 208 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.115338:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x190:0x0].0x0 (ffff89b681690200) 00010000:00010000:2.0:1769074799.116472:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5400/0x7fe2323adfd3685d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x18f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878eb34 expref: 207 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116492:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x18f:0x0].0x0 (ffff89b6821d6000) 00010000:00010000:2.0:1769074799.116505:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d800/0x7fe2323adfd36649 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x18c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ea8c expref: 206 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116738:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3800/0x7fe2323adfd3658c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x18c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ea4d expref: 205 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116750:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x18c:0x0].0x0 (ffff89b665617d00) 00010000:00010000:2.0:1769074799.116764:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3da00/0x7fe2323adfd36298 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x18b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e958 expref: 204 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116773:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x18b:0x0].0x0 (ffff89b681690100) 00010000:00010000:2.0:1769074799.116783:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3de00/0x7fe2323adfd36092 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x18a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e8b0 expref: 203 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116791:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x18a:0x0].0x0 (ffff89b666b4ab00) 00010000:00010000:2.0:1769074799.116801:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5e00/0x7fe2323adfd35ed9 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x189:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e81d expref: 202 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116835:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x189:0x0].0x0 (ffff89b665618c00) 00010000:00010000:2.0:1769074799.116849:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97200/0x7fe2323adfd359c3 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x185:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e679 expref: 201 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.116858:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x185:0x0].0x0 (ffff89b666b4a300) 00010000:00010000:2.0:1769074799.117058:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447800/0x7fe2323adfd3589d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x182:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e610 expref: 200 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117073:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd3577e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x182:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e5b5 expref: 199 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117093:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x182:0x0].0x0 (ffff89b6821d6d00) 00010000:00010000:2.0:1769074799.117107:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a600/0x7fe2323adfd35483 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x181:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e4b9 expref: 198 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117116:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x181:0x0].0x0 (ffff89b665618500) 00010000:00010000:2.0:1769074799.117126:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97600/0x7fe2323adfd352d8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x17f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e42d expref: 197 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117134:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x17f:0x0].0x0 (ffff89b681690d00) 00010000:00010000:2.0:1769074799.117144:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447400/0x7fe2323adfd34e2b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x178:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e2ba expref: 196 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117156:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053800/0x7fe2323adfd34ce9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x178:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e25f expref: 195 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117164:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x178:0x0].0x0 (ffff89b666b4ad00) 00010000:00010000:2.0:1769074799.117175:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5e00/0x7fe2323adfd34ac0 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x175:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e194 expref: 194 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117186:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5000/0x7fe2323adfd3498c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x175:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e140 expref: 193 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117193:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x175:0x0].0x0 (ffff89b543631600) 00010000:00010000:2.0:1769074799.117202:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437000/0x7fe2323adfd34652 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x174:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878e021 expref: 192 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117210:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x174:0x0].0x0 (ffff89b665618600) 00010000:00010000:2.0:1769074799.117219:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5200/0x7fe2323adfd34453 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x172:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878df79 expref: 191 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117229:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x172:0x0].0x0 (ffff89b665618400) 00010000:00010000:2.0:1769074799.117284:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436800/0x7fe2323adfd34350 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x16f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878df2c expref: 190 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117296:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1200/0x7fe2323adfd3423f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x16f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ded1 expref: 189 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117304:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x16f:0x0].0x0 (ffff89b643a80300) 00010000:00010000:2.0:1769074799.117313:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031200/0x7fe2323adfd33e72 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x16e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878dda4 expref: 188 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117321:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x16e:0x0].0x0 (ffff89b67a318500) 00010000:00010000:2.0:1769074799.117329:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46200/0x7fe2323adfd33cf8 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x16d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878dd26 expref: 187 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.117339:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x16d:0x0].0x0 (ffff89b67a318a00) 00010000:00010000:2.0:1769074799.118202:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4600/0x7fe2323adfd33ba1 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x16a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878dca8 expref: 186 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118224:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b640030000/0x7fe2323adfd33a97 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x16a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878dc54 expref: 185 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118233:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x16a:0x0].0x0 (ffff89b543631f00) 00010000:00010000:2.0:1769074799.118244:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3dc00/0x7fe2323adfd337aa lrc: 0/0,0 mode: --/CR res: [0x200000405:0x169:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878db5f expref: 184 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118252:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x169:0x0].0x0 (ffff89b6821d6f00) 00010000:00010000:2.0:1769074799.118292:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b640031a00/0x7fe2323adfd33431 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x166:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878da2b expref: 183 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118303:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x166:0x0].0x0 (ffff89b64ba54f00) 00010000:00010000:2.0:1769074799.118314:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4c00/0x7fe2323adfd332fd lrc: 0/0,0 mode: --/CR res: [0x200000405:0x163:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d9c2 expref: 182 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118330:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4400/0x7fe2323adfd331de lrc: 0/0,0 mode: --/PR res: [0x200000405:0x163:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d960 expref: 181 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118741:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x163:0x0].0x0 (ffff89b64ba54600) 00010000:00010000:2.0:1769074799.118767:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47a00/0x7fe2323adfd32ec0 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x161:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d848 expref: 180 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.118778:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x161:0x0].0x0 (ffff89b6519abb00) 00010000:00010000:2.0:1769074799.119044:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031a00/0x7fe2323adfd32d00 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x15f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d7b5 expref: 179 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119058:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x15f:0x0].0x0 (ffff89b6821d6700) 00010000:00010000:2.0:1769074799.119074:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46000/0x7fe2323adfd32c27 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x15b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d76f expref: 178 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119213:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70600/0x7fe2323adfd32ade lrc: 0/0,0 mode: --/PR res: [0x200000405:0x15b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d714 expref: 177 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119224:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x15b:0x0].0x0 (ffff89b665616300) 00010000:00010000:2.0:1769074799.119238:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925400/0x7fe2323adfd32814 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x157:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d626 expref: 176 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119251:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030a00/0x7fe2323adfd3272d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x157:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d5d2 expref: 175 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119259:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x157:0x0].0x0 (ffff89b6480cf400) 00010000:00010000:2.0:1769074799.119269:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d200/0x7fe2323adfd32543 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x154:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d531 expref: 174 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119280:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031800/0x7fe2323adfd323fa lrc: 0/0,0 mode: --/PR res: [0x200000405:0x154:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d4d6 expref: 173 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119289:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x154:0x0].0x0 (ffff89b6472cb100) 00010000:00010000:2.0:1769074799.119298:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b640030800/0x7fe2323adfd3211b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x152:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d3da expref: 172 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119307:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x152:0x0].0x0 (ffff89b665617800) 00010000:00010000:2.0:1769074799.119638:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030600/0x7fe2323adfd32018 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x14f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d38d expref: 171 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119658:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70a00/0x7fe2323adfd31ed6 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x14f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d332 expref: 170 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119668:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x14f:0x0].0x0 (ffff89b64a219c00) 00010000:00010000:2.0:1769074799.119680:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd319ea lrc: 0/0,0 mode: --/CR res: [0x200000405:0x14c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d18e expref: 169 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119688:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x14c:0x0].0x0 (ffff89b64ba54500) 00010000:00010000:2.0:1769074799.119698:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71800/0x7fe2323adfd31934 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x149:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d156 expref: 168 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119709:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924000/0x7fe2323adfd317eb lrc: 0/0,0 mode: --/PR res: [0x200000405:0x149:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878d102 expref: 167 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119716:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x149:0x0].0x0 (ffff89b5410b7500) 00010000:00010000:2.0:1769074799.119724:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925800/0x7fe2323adfd3137d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x145:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878cf7a expref: 166 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119734:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71200/0x7fe2323adfd31273 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x145:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878cf34 expref: 165 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119741:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x145:0x0].0x0 (ffff89b643dacc00) 00010000:00010000:2.0:1769074799.119750:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0600/0x7fe2323adfd30d64 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x142:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878cd74 expref: 164 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119757:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x142:0x0].0x0 (ffff89b66ebf8600) 00010000:00010000:2.0:1769074799.119766:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436a00/0x7fe2323adfd30c37 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x13f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878cd04 expref: 163 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119778:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436000/0x7fe2323adfd30b42 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x13f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ccb7 expref: 162 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.119786:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x13f:0x0].0x0 (ffff89b665617200) 00010000:00010000:2.0:1769074799.119795:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437c00/0x7fe2323adfd3080f lrc: 0/0,0 mode: --/CR res: [0x200000405:0x13e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878cbb4 expref: 161 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120027:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x13e:0x0].0x0 (ffff89b6519ab900) 00010000:00010000:2.0:1769074799.120046:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0a00/0x7fe2323adfd3008a lrc: 0/0,0 mode: --/CR res: [0x200000405:0x135:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c968 expref: 160 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120060:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2600/0x7fe2323adfd2ff6b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x135:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c8ff expref: 159 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120067:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x135:0x0].0x0 (ffff89b6539b1c00) 00010000:00010000:2.0:1769074799.120077:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3200/0x7fe2323adfd2fb20 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x130:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c79a expref: 158 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120516:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96200/0x7fe2323adfd2f9f3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x130:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c738 expref: 157 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120618:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x130:0x0].0x0 (ffff89b665618900) 00010000:00010000:2.0:1769074799.120634:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436c00/0x7fe2323adfd2f6ce lrc: 0/0,0 mode: --/CR res: [0x200000405:0x12f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c62e expref: 156 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120644:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x12f:0x0].0x0 (ffff89b665618b00) 00010000:00010000:2.0:1769074799.120655:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b436600/0x7fe2323adfd2f4b3 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x12e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c57f expref: 155 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120662:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x12e:0x0].0x0 (ffff89b665618100) 00010000:00010000:2.0:1769074799.120672:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651200/0x7fe2323adfd2ef65 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x128:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c3cd expref: 154 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120684:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a000/0x7fe2323adfd2ee5b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x128:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c387 expref: 153 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120692:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x128:0x0].0x0 (ffff89b665618f00) 00010000:00010000:2.0:1769074799.120702:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70e00/0x7fe2323adfd2e785 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x122:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c149 expref: 152 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120714:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437200/0x7fe2323adfd2e6e4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x122:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878c11f expref: 151 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120722:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x122:0x0].0x0 (ffff89b543631400) 00010000:00010000:2.0:1769074799.120732:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71600/0x7fe2323adfd2e28b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x11d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878bfc1 expref: 150 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120741:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2200/0x7fe2323adfd2e173 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x11d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878bf6d expref: 149 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120749:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x11d:0x0].0x0 (ffff89b6821d6100) 00010000:00010000:2.0:1769074799.120759:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2c00/0x7fe2323adfd2de24 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x11c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878be4e expref: 148 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120766:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x11c:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074799.120778:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47200/0x7fe2323adfd2dc5d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x11b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878bdc2 expref: 147 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.120786:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x11b:0x0].0x0 (ffff89b6821d6500) 00010000:00010000:2.0:1769074799.120796:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b437800/0x7fe2323adfd2db29 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x118:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878bd59 expref: 146 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121243:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925e00/0x7fe2323adfd2da0a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x118:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878bcfe expref: 145 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121259:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x118:0x0].0x0 (ffff89b6480cfb00) 00010000:00010000:2.0:1769074799.121272:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd2d716 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x116:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878bbfb expref: 144 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121283:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x116:0x0].0x0 (ffff89b6480cf700) 00010000:00010000:2.0:1769074799.121291:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924e00/0x7fe2323adfd2d388 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x113:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878badc expref: 143 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121299:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x113:0x0].0x0 (ffff89b681690c00) 00010000:00010000:2.0:1769074799.121309:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71c00/0x7fe2323adfd2d14a lrc: 0/0,0 mode: --/CR res: [0x200000405:0x10e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ba2d expref: 142 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121321:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd2d02b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x10e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b9c4 expref: 141 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121329:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x10e:0x0].0x0 (ffff89b6480cfd00) 00010000:00010000:2.0:1769074799.121339:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5000/0x7fe2323adfd2caeb lrc: 0/0,0 mode: --/CR res: [0x200000405:0x10b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b812 expref: 140 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121431:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x10b:0x0].0x0 (ffff89b64a219800) 00010000:00010000:2.0:1769074799.121450:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4c00/0x7fe2323adfd2ca7b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x108:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b7e1 expref: 139 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121465:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70200/0x7fe2323adfd2c94e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x108:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b78d expref: 138 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121474:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x108:0x0].0x0 (ffff89b64a219900) 00010000:00010000:2.0:1769074799.121484:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47800/0x7fe2323adfd2c056 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x102:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b48b expref: 137 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.121492:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x102:0x0].0x0 (ffff89b64a19d800) 00010000:00010000:2.0:1769074799.122216:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f70800/0x7fe2323adfd2bfbc lrc: 0/0,0 mode: --/CR res: [0x200000405:0xff:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b45a expref: 136 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122232:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46e00/0x7fe2323adfd2be57 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xff:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b3f1 expref: 135 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122241:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xff:0x0].0x0 (ffff89b543631a00) 00010000:00010000:2.0:1769074799.122273:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46c00/0x7fe2323adfd2ba4b lrc: 0/0,0 mode: --/CR res: [0x200000405:0xfa:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b285 expref: 134 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122285:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5a00/0x7fe2323adfd2b93a lrc: 0/0,0 mode: --/PR res: [0x200000405:0xfa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b23f expref: 133 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122293:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xfa:0x0].0x0 (ffff89b543631c00) 00010000:00010000:2.0:1769074799.122301:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547f71e00/0x7fe2323adfd2b631 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xf9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878b135 expref: 132 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122309:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xf9:0x0].0x0 (ffff89b543631000) 00010000:00010000:2.0:1769074799.122318:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3a00/0x7fe2323adfd2b1a0 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xf3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878afc2 expref: 131 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122328:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2e00/0x7fe2323adfd2b08f lrc: 0/0,0 mode: --/PR res: [0x200000405:0xf3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878af7c expref: 130 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122337:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xf3:0x0].0x0 (ffff89b5410b7d00) 00010000:00010000:2.0:1769074799.122649:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925c00/0x7fe2323adfd2ab48 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xf0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878adbc expref: 129 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122665:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xf0:0x0].0x0 (ffff89b5410b7000) 00010000:00010000:2.0:1769074799.122679:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924200/0x7fe2323adfd2a99d lrc: 0/0,0 mode: --/CR res: [0x200000405:0xee:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ad30 expref: 128 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122688:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xee:0x0].0x0 (ffff89b643a80800) 00010000:00010000:2.0:1769074799.122698:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff3600/0x7fe2323adfd2a75f lrc: 0/0,0 mode: --/CR res: [0x200000405:0xe9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ac81 expref: 127 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122709:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0200/0x7fe2323adfd2a5f3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xe9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ac0a expref: 126 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122717:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xe9:0x0].0x0 (ffff89b681690e00) 00010000:00010000:2.0:1769074799.122727:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0800/0x7fe2323adfd2a34c lrc: 0/0,0 mode: --/CR res: [0x200000405:0xe6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878ab2a expref: 125 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122738:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1000/0x7fe2323adfd2a23b lrc: 0/0,0 mode: --/PR res: [0x200000405:0xe6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878aad6 expref: 124 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122747:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xe6:0x0].0x0 (ffff89b681697900) 00010000:00010000:2.0:1769074799.122755:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679924800/0x7fe2323adfd29bff lrc: 0/0,0 mode: --/CR res: [0x200000405:0xe2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a8d7 expref: 123 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122764:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xe2:0x0].0x0 (ffff89b6542eb300) 00010000:00010000:2.0:1769074799.122773:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96a00/0x7fe2323adfd29824 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xe0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a795 expref: 122 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122782:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xe0:0x0].0x0 (ffff89b5410b7e00) 00010000:00010000:2.0:1769074799.122791:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd29610 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xdf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a6ed expref: 121 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.122799:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xdf:0x0].0x0 (ffff89b5410b7900) 00010000:00010000:2.0:1769074799.123037:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447600/0x7fe2323adfd28ff0 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xda:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a4f5 expref: 120 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123049:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xda:0x0].0x0 (ffff89b64a219400) 00010000:00010000:2.0:1769074799.123062:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0c00/0x7fe2323adfd28f72 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xd7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a4cb expref: 119 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123073:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0000/0x7fe2323adfd28e06 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xd7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a45b expref: 118 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123179:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xd7:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074799.123196:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b679925600/0x7fe2323adfd2891a lrc: 0/0,0 mode: --/CR res: [0x200000405:0xd3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a2be expref: 117 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123205:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xd3:0x0].0x0 (ffff89b6439add00) 00010000:00010000:2.0:1769074799.123216:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1c00/0x7fe2323adfd28817 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xd0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a26a expref: 116 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123232:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446600/0x7fe2323adfd286f1 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xd0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878a208 expref: 115 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123240:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xd0:0x0].0x0 (ffff89b681697b00) 00010000:00010000:2.0:1769074799.123250:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2a00/0x7fe2323adfd28006 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xcc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789fd1 expref: 114 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123258:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xcc:0x0].0x0 (ffff89b6503c5200) 00010000:00010000:2.0:1769074799.123266:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1600/0x7fe2323adfd27e1c lrc: 0/0,0 mode: --/CR res: [0x200000405:0xcb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789f37 expref: 113 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123274:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xcb:0x0].0x0 (ffff89b665617700) 00010000:00010000:2.0:1769074799.123287:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea1a00/0x7fe2323adfd27c2b lrc: 0/0,0 mode: --/PR res: [0x200000405:0xca:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789e96 expref: 112 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123295:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xca:0x0].0x0 (ffff89b665617000) 00010000:00010000:2.0:1769074799.123303:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea0e00/0x7fe2323adfd27aaa lrc: 0/0,0 mode: --/CR res: [0x200000405:0xc8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789e18 expref: 111 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123311:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xc8:0x0].0x0 (ffff89b665617400) 00010000:00010000:2.0:1769074799.123319:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b64eff2000/0x7fe2323adfd2796f lrc: 0/0,0 mode: --/CR res: [0x200000405:0xc5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789da8 expref: 110 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123330:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d400/0x7fe2323adfd27850 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xc5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789d4d expref: 109 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123339:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xc5:0x0].0x0 (ffff89b67a317a00) 00010000:00010000:2.0:1769074799.123435:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b640030c00/0x7fe2323adfd2759b lrc: 0/0,0 mode: --/CR res: [0x200000405:0xc3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789c66 expref: 108 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123446:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xc3:0x0].0x0 (ffff89b6503c5600) 00010000:00010000:2.0:1769074799.123456:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b640031000/0x7fe2323adfd27498 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789c0b expref: 107 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123469:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd2736b lrc: 0/0,0 mode: --/PR res: [0x200000405:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789ba9 expref: 106 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123477:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xc0:0x0].0x0 (ffff89b665617500) 00010000:00010000:2.0:1769074799.123488:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b08fc00/0x7fe2323adfd27054 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789aad expref: 105 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123664:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xbe:0x0].0x0 (ffff89b665618800) 00010000:00010000:2.0:1769074799.123681:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3ce00/0x7fe2323adfd26b22 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789910 expref: 104 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123690:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xba:0x0].0x0 (ffff89b665618300) 00010000:00010000:2.0:1769074799.123701:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030e00/0x7fe2323adfd26a18 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987898bc expref: 103 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123712:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd268f9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878986f expref: 102 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123722:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xb7:0x0].0x0 (ffff89b6480cfa00) 00010000:00010000:2.0:1769074799.123732:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3ca00/0x7fe2323adfd26660 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878978f expref: 101 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123750:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676031000/0x7fe2323adfd26548 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789742 expref: 100 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123759:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xb4:0x0].0x0 (ffff89b665615f00) 00010000:00010000:2.0:1769074799.123770:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030c00/0x7fe2323adfd26104 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987895cf expref: 99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123781:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb400/0x7fe2323adfd25ff3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878957b expref: 98 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.123788:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xaf:0x0].0x0 (ffff89b67a317600) 00010000:00010000:2.0:1769074799.123798:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b000/0x7fe2323adfd25dae lrc: 0/0,0 mode: --/CR res: [0x200000405:0xac:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987894b0 expref: 97 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124028:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030200/0x7fe2323adfd25c81 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789447 expref: 96 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124221:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xac:0x0].0x0 (ffff89b665611000) 00010000:00010000:2.0:1769074799.124238:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b676030000/0x7fe2323adfd25a2e lrc: 0/0,0 mode: --/CR res: [0x200000405:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878938a expref: 95 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124249:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6c00/0x7fe2323adfd25908 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789328 expref: 94 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124257:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xa9:0x0].0x0 (ffff89b5410b7400) 00010000:00010000:2.0:1769074799.124267:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491faa00/0x7fe2323adfd256d1 lrc: 0/0,0 mode: --/CR res: [0x200000405:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878926b expref: 93 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124284:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6000/0x7fe2323adfd255c7 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878921e expref: 92 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124293:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xa6:0x0].0x0 (ffff89b5410b7100) 00010000:00010000:2.0:1769074799.124303:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4800/0x7fe2323adfd252fd lrc: 0/0,0 mode: --/CR res: [0x200000405:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789130 expref: 91 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124311:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xa4:0x0].0x0 (ffff89b5410b7600) 00010000:00010000:2.0:1769074799.124320:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b4e00/0x7fe2323adfd251fa lrc: 0/0,0 mode: --/CR res: [0x200000405:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987890dc expref: 90 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124330:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5a00/0x7fe2323adfd250b1 lrc: 0/0,0 mode: --/PR res: [0x200000405:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198789081 expref: 89 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124339:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xa1:0x0].0x0 (ffff89b6480cf800) 00010000:00010000:2.0:1769074799.124436:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44800/0x7fe2323adfd24daf lrc: 0/0,0 mode: --/PR res: [0x200000405:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788f77 expref: 88 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124446:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0xa0:0x0].0x0 (ffff89b665611100) 00010000:00010000:2.0:1769074799.124456:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b6725b5c00/0x7fe2323adfd24bda lrc: 0/0,0 mode: --/CR res: [0x200000405:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788ee4 expref: 87 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124467:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45e00/0x7fe2323adfd24afa lrc: 0/0,0 mode: --/PR res: [0x200000405:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788e9e expref: 86 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124474:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x9d:0x0].0x0 (ffff89b653a0d100) 00010000:00010000:2.0:1769074799.124484:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719600/0x7fe2323adfd2481b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788db0 expref: 85 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124657:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x9b:0x0].0x0 (ffff89b64a19d700) 00010000:00010000:2.0:1769074799.124679:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54b719a00/0x7fe2323adfd24726 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x98:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788d63 expref: 84 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124690:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd2461c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x98:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788d16 expref: 83 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124698:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x98:0x0].0x0 (ffff89b653a0d500) 00010000:00010000:2.0:1769074799.124707:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45000/0x7fe2323adfd2409d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788b4f expref: 82 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124714:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x95:0x0].0x0 (ffff89b681690600) 00010000:00010000:2.0:1769074799.124723:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a400/0x7fe2323adfd24018 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x91:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788b25 expref: 81 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124736:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44a00/0x7fe2323adfd23eba lrc: 0/0,0 mode: --/PR res: [0x200000405:0x91:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788abc expref: 80 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124744:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x91:0x0].0x0 (ffff89b6472cb000) 00010000:00010000:2.0:1769074799.124753:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053200/0x7fe2323adfd237cf lrc: 0/0,0 mode: --/PR res: [0x200000405:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788885 expref: 79 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124761:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x8d:0x0].0x0 (ffff89b6539b1a00) 00010000:00010000:2.0:1769074799.124771:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c000/0x7fe2323adfd23774 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788870 expref: 78 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124781:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd2366a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878880e expref: 77 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.124789:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x8a:0x0].0x0 (ffff89b6539b1500) 00010000:00010000:2.0:1769074799.124798:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd232f1 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x88:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987886e8 expref: 76 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125042:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x88:0x0].0x0 (ffff89b6539b1700) 00010000:00010000:2.0:1769074799.125058:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117800/0x7fe2323adfd22f47 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987885b4 expref: 75 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125068:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x86:0x0].0x0 (ffff89b643dac800) 00010000:00010000:2.0:1769074799.125077:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446c00/0x7fe2323adfd22d64 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788513 expref: 74 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125197:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x84:0x0].0x0 (ffff89b643dace00) 00010000:00010000:2.0:1769074799.125212:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c600/0x7fe2323adfd22c61 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x81:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987884b8 expref: 73 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125224:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d000/0x7fe2323adfd22b57 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x81:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198788472 expref: 72 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125233:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x81:0x0].0x0 (ffff89b665618200) 00010000:00010000:2.0:1769074799.125244:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c200/0x7fe2323adfd22656 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987882dc expref: 71 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125252:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x7f:0x0].0x0 (ffff89b67a318b00) 00010000:00010000:2.0:1769074799.125262:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97a00/0x7fe2323adfd22186 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878813f expref: 70 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125275:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x7b:0x0].0x0 (ffff89b665611300) 00010000:00010000:2.0:1769074799.125284:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd21cb6 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x75:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787f8d expref: 69 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125295:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd21b74 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x75:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787f39 expref: 68 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125301:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x75:0x0].0x0 (ffff89b665616600) 00010000:00010000:2.0:1769074799.125312:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd21706 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x70:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787db8 expref: 67 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125322:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd21626 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x70:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787d72 expref: 66 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125330:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x70:0x0].0x0 (ffff89b643daca00) 00010000:00010000:2.0:1769074799.125339:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd21141 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787be3 expref: 65 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125429:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x6e:0x0].0x0 (ffff89b6539b1600) 00010000:00010000:2.0:1769074799.125442:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3c400/0x7fe2323adfd20f65 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787b42 expref: 64 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125450:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x6c:0x0].0x0 (ffff89b64a19da00) 00010000:00010000:2.0:1769074799.125459:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd20e3f lrc: 0/0,0 mode: --/CR res: [0x200000405:0x69:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787ae7 expref: 63 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125471:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052600/0x7fe2323adfd20d2e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x69:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787a9a expref: 62 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.125479:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x69:0x0].0x0 (ffff89b67a318200) 00010000:00010000:2.0:1769074799.125488:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd209a7 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x64:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787974 expref: 61 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126212:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd2085e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x64:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787912 expref: 60 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126224:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x64:0x0].0x0 (ffff89b6429cd800) 00010000:00010000:2.0:1769074799.126236:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd20555 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x63:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787808 expref: 59 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126246:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x63:0x0].0x0 (ffff89b67a317900) 00010000:00010000:2.0:1769074799.126256:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96800/0x7fe2323adfd20038 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x5d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878765d expref: 58 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126269:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd1ff35 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x5d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787610 expref: 57 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126279:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x5d:0x0].0x0 (ffff89b6480cfc00) 00010000:00010000:2.0:1769074799.126290:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5800/0x7fe2323adfd1fc79 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x5b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787522 expref: 56 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126299:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x5b:0x0].0x0 (ffff89b65436a800) 00010000:00010000:2.0:1769074799.126309:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447200/0x7fe2323adfd1f890 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x57:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987873e7 expref: 55 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126320:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97e00/0x7fe2323adfd1f7c5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x57:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987873a8 expref: 54 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126328:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x57:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:2.0:1769074799.126338:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447c00/0x7fe2323adfd1f4ca lrc: 0/0,0 mode: --/CR res: [0x200000405:0x54:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987872b3 expref: 53 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126442:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447e00/0x7fe2323adfd1f3f8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x54:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878726d expref: 52 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126449:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x54:0x0].0x0 (ffff89b6543ea000) 00010000:00010000:2.0:1769074799.126460:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd1f0da lrc: 0/0,0 mode: --/CR res: [0x200000405:0x51:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878716a expref: 51 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126470:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5c00/0x7fe2323adfd1f047 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x51:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198787140 expref: 50 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126478:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x51:0x0].0x0 (ffff89b665617900) 00010000:00010000:2.0:1769074799.126490:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd1eb31 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786f9c expref: 49 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126660:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x4f:0x0].0x0 (ffff89b67a318700) 00010000:00010000:2.0:1769074799.126675:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b447a00/0x7fe2323adfd1e9a2 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786f1e expref: 48 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126683:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x4d:0x0].0x0 (ffff89b6436c4600) 00010000:00010000:2.0:1769074799.126694:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446400/0x7fe2323adfd1e8a6 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x4a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786eca expref: 47 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126704:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4200/0x7fe2323adfd1e75d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x4a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786e6f expref: 46 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126713:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x4a:0x0].0x0 (ffff89b67a317b00) 00010000:00010000:2.0:1769074799.126723:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96c00/0x7fe2323adfd1e4f5 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x47:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786d96 expref: 45 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126734:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b640030400/0x7fe2323adfd1e3e4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x47:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786d42 expref: 44 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126743:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x47:0x0].0x0 (ffff89b67a318000) 00010000:00010000:2.0:1769074799.126753:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052800/0x7fe2323adfd1dfdf lrc: 0/0,0 mode: --/CR res: [0x200000405:0x42:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786beb expref: 43 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126764:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea5600/0x7fe2323adfd1dece lrc: 0/0,0 mode: --/PR res: [0x200000405:0x42:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786b97 expref: 42 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126772:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x42:0x0].0x0 (ffff89b65436a900) 00010000:00010000:2.0:1769074799.126783:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052c00/0x7fe2323adfd1dc43 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x3f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786ac5 expref: 41 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126794:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97c00/0x7fe2323adfd1db32 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x3f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786a71 expref: 40 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.126802:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x3f:0x0].0x0 (ffff89b64a219000) 00010000:00010000:2.0:1769074799.127048:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa600/0x7fe2323adfd1d654 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x3c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987868db expref: 39 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127058:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x3c:0x0].0x0 (ffff89b64a19de00) 00010000:00010000:2.0:1769074799.127067:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052400/0x7fe2323adfd1d44e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878683a expref: 38 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127076:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x3a:0x0].0x0 (ffff89b643dac700) 00010000:00010000:2.0:1769074799.127198:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa000/0x7fe2323adfd1d3f3 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786825 expref: 37 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127211:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b052200/0x7fe2323adfd1d2bf lrc: 0/0,0 mode: --/PR res: [0x200000405:0x37:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987867b5 expref: 36 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127220:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x37:0x0].0x0 (ffff89b643dac900) 00010000:00010000:2.0:1769074799.127230:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053600/0x7fe2323adfd1cf46 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878668f expref: 35 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127239:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x36:0x0].0x0 (ffff89b6542ebc00) 00010000:00010000:2.0:1769074799.127249:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fea4800/0x7fe2323adfd1ce35 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x33:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786634 expref: 34 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127260:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fa200/0x7fe2323adfd1cd2b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x33:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987865e7 expref: 33 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127267:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x33:0x0].0x0 (ffff89b681697200) 00010000:00010000:2.0:1769074799.127277:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b7400/0x7fe2323adfd1c815 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x30:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878644a expref: 32 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127286:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x30:0x0].0x0 (ffff89b643dac600) 00010000:00010000:2.0:1769074799.127297:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d96400/0x7fe2323adfd1c782 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786419 expref: 31 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127309:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547d97000/0x7fe2323adfd1c624 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987863b0 expref: 30 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127318:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x2d:0x0].0x0 (ffff89b653a0d600) 00010000:00010000:2.0:1769074799.127327:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44400/0x7fe2323adfd1c31b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x2c:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878629f expref: 29 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127336:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x2c:0x0].0x0 (ffff89b643dacd00) 00010000:00010000:2.0:1769074799.127437:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba44c00/0x7fe2323adfd1c1fc lrc: 0/0,0 mode: --/CR res: [0x200000405:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878624b expref: 28 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127450:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b053400/0x7fe2323adfd1c0f2 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198786205 expref: 27 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127458:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x28:0x0].0x0 (ffff89b643a80e00) 00010000:00010000:2.0:1769074799.127467:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fac00/0x7fe2323adfd1bdcd lrc: 0/0,0 mode: --/PR res: [0x200000405:0x27:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860df expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127473:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x27:0x0].0x0 (ffff89b6543ea500) 00010000:00010000:2.0:1769074799.127481:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5491fb000/0x7fe2323adfd1bd5d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x25:0x0].0x0 bits 0xd/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860ca expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127491:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546116c00/0x7fe2323adfd1bd33 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860bc expref: 24 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127516:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117400/0x7fe2323adfd1bd09 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x25:0x0].0x0 bits 0xa/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860ae expref: 23 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127685:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b546117200/0x7fe2323adfd1bced lrc: 0/0,0 mode: --/PR res: [0x200000405:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860a0 expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127699:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x25:0x0].0x0 (ffff89b64a19db00) 00010000:00010000:2.0:1769074799.127710:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651400/0x7fe2323adfd1ba93 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785fea expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127720:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x22:0x0].0x0 (ffff89b54362b200) 00010000:00010000:2.0:1769074799.127729:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641651800/0x7fe2323adfd1b6a3 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x43:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785e8c expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127737:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x43:0x0].0x0 (ffff89b665611500) 00010000:00010000:2.0:1769074799.127747:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b54ba45c00/0x7fe2323adfd1b656 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785e7e expref: 19 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127759:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650400/0x7fe2323adfd1b633 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785e77 expref: 18 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127769:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1e:0x0].0x0 (ffff89b54362bb00) 00010000:00010000:2.0:1769074799.127779:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b5481b6e00/0x7fe2323adfd1b2f9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785d4a expref: 17 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127789:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1d:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:2.0:1769074799.127798:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641650000/0x7fe2323adfd1ada4 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785b98 expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.127982:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x3c:0x0].0x0 (ffff89b665611e00) 00010000:00010000:2.0:1769074799.127991:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074799.128153:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e666850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:39339:x1855007616961536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 17822us (18510us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074799.128164:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1035 00000100:00100000:3.0:1769074799.211121:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074799.211221:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074799.211338:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961792 00000100:00100000:2.0:1769074799.211347:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961792 00000100:00100000:2.0:1769074799.211431:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1036 00000100:00100000:2.0:1769074799.211441:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:60220:x1855007616961792:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074799.211458:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 21 locks, starting at 0 00010000:00010000:2.0:1769074799.211502:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd5472e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c354 expref: 26 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211537:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba000/0x7fe2323adfd54727 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c346 expref: 25 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211558:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54735 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c34d expref: 24 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211592:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b052a00/0x7fe2323adfd54567 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c27b expref: 23 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.211612:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cdea600/0x7fe2323adfd54704 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c338 expref: 22 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211631:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54cdeba00/0x7fe2323adfd546f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c32a expref: 21 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211651:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876200/0x7fe2323adfd54694 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c307 expref: 20 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211688:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54910c200/0x7fe2323adfd54686 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c300 expref: 19 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.211707:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67b446800/0x7fe2323adfd54671 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2eb expref: 18 pid: 28519 timeout: 0 lvb_type: 1 00002000:00010000:2.0:1769074799.211760:0:8424:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x33:0x0] updating lvb size from disk: 0 -> 69287 00002000:00010000:2.0:1769074799.211763:0:8424:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x33:0x0] updating lvb mtime from disk: 0 -> 1769074604 00002000:00010000:2.0:1769074799.211766:0:8424:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x33:0x0] updating lvb ctime from disk: 0 -> 1769074604 00002000:00010000:2.0:1769074799.211768:0:8424:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x33:0x0] updating lvb blocks from disk: 0 -> 144 00010000:00010000:2.0:1769074799.211777:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b676031c00/0x7fe2323adfd396e0 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x33:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619878fb48 expref: 17 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.211884:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547876e00/0x7fe2323adfd48cab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879570c expref: 16 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.211920:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd48ca4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795705 expref: 15 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.211959:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd48c96 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987956fe expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.211990:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd48699 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x34:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795553 expref: 13 pid: 28519 timeout: 0 lvb_type: 1 00002000:00010000:2.0:1769074799.212010:0:8424:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x89:0x0] updating lvb size from disk: 0 -> 1516 00002000:00010000:2.0:1769074799.212013:0:8424:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x89:0x0] updating lvb mtime from disk: 1769074420 -> 1769074421 00002000:00010000:2.0:1769074799.212015:0:8424:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x89:0x0] updating lvb atime from disk: 0 -> 1769074421 00002000:00010000:2.0:1769074799.212017:0:8424:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x89:0x0] updating lvb ctime from disk: 1769074420 -> 1769074421 00002000:00010000:2.0:1769074799.212019:0:8424:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x89:0x0] updating lvb blocks from disk: 0 -> 8 00010000:00010000:2.0:1769074799.212025:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b54ba44e00/0x7fe2323adfd1bd25 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801080000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987860b5 expref: 12 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.212041:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fbc00/0x7fe2323adfd1ba1c lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x32:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785fc7 expref: 11 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.212058:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fb200/0x7fe2323adfd1b61e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x85:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785e70 expref: 10 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.212076:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5491fa800/0x7fe2323adfd1b609 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x86:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785e69 expref: 9 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.212107:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b7600/0x7fe2323adfd1b17f lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x31:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785ce8 expref: 8 pid: 9048 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074799.212149:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b7000/0x7fe2323adfd1a9a6 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x2c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785a6b expref: 7 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.212177:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b5481b7800/0x7fe2323adfd18151 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198784b84 expref: 6 pid: 10284 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074799.212193:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074799.212252:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:60220:x1855007616961792:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 812us (1029us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074799.212259:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1036 00000100:00100000:1.0:1769074799.212992:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961664 00000100:00100000:1.0:1769074799.213003:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961664 00002000:00100000:1.0:1769074799.213042:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x2c1:0x0] (0->2097151) req@ffff89b549f2a300 x1855007616961664/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:335/0 lens 488/0 e 0 to 0 dl 1769074810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074799.213051:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2a300 with x1855007616961664 ext(0->2097151) 00002000:00010000:1.0:1769074799.213056:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f2a300 00000100:00100000:1.0:1769074799.213080:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8736 00000100:00100000:1.0:1769074799.213088:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2407:x1855007616961664:12345-192.168.204.60@tcp:4:dd.0 00000100:00100000:1.0:1769074799.256897:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007616961664:12345-192.168.204.60@tcp:4:dd.0 Request processed in 43798us (45773us total) trans 21474841265 rc 0/0 00000100:00100000:1.0:1769074799.256913:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8736 00002000:00100000:1.0:1769074799.256927:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x2c1:0x0] (0->2097151) req@ffff89b549f2a300 x1855007616961664/t21474841265(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:335/0 lens 488/448 e 0 to 0 dl 1769074810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074799.256939:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b549f2a300 with x1855007616961664 ext(0->2097151) 00002000:00010000:1.0:1769074799.256946:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b549f2a300 00000100:00100000:1.0:1769074799.278123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074799.278220:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616961920 00000100:00100000:0.0:1769074799.278231:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616961920 00000100:00100000:0.0:1769074799.278302:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1037 00000100:00100000:0.0:1769074799.278313:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:46917:x1855007616961920:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074799.278330:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 16 locks, starting at 0 00002000:00010000:0.0:1769074799.278349:0:6498:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:0.0:1769074799.278352:0:6498:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb mtime from disk: 0 -> 1769074798 00002000:00010000:0.0:1769074799.278354:0:6498:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb ctime from disk: 0 -> 1769074798 00002000:00010000:0.0:1769074799.278356:0:6498:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb blocks from disk: 0 -> 4096 00010000:00010000:0.0:1769074799.278384:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88400/0x7fe2323adfd547f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3e7 expref: 23 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278425:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54598 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->33554431) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c29e expref: 22 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278446:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea200/0x7fe2323adfd5470b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4194304->16777215) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c33f expref: 21 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278464:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea800/0x7fe2323adfd546fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3145728->4194303) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c331 expref: 20 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278481:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cdea400/0x7fe2323adfd546ef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->3145727) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c323 expref: 19 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278498:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54cdeaa00/0x7fe2323adfd546e8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c31c expref: 18 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278530:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547f0f000/0x7fe2323adfd5468d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->8388607) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c30e expref: 17 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278554:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b682164000/0x7fe2323adfd5467f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2f9 expref: 16 pid: 8399 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278572:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547876a00/0x7fe2323adfd54678 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c2f2 expref: 15 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278590:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877200/0x7fe2323adfd48cb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795713 expref: 14 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278608:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd48c8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf61987956f7 expref: 13 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278626:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547877c00/0x7fe2323adfd486ae lrc: 0/0,0 mode: --/PR res: [0x280000400:0x34:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198795561 expref: 12 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074799.278645:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba45400/0x7fe2323adfd1ba2a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x33:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198785fce expref: 11 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278680:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b54ba44200/0x7fe2323adfd18189 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198784b99 expref: 10 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278698:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b6600/0x7fe2323adfd17eb8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198784a9d expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278717:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b5481b7a00/0x7fe2323adfd17eb1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf6198784a96 expref: 8 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074799.278727:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074799.278793:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:46917:x1855007616961920:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 482us (690us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074799.278801:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1037 00000100:00100000:0.0:1769074799.516973:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625197440:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074799.517045:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074799.518798:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197440 00000100:00100000:1.0:1769074799.518829:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197440 00000100:00100000:1.0:1769074799.518895:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2664 00000100:00100000:1.0:1769074799.518904:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625197440:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074799.518943:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00100000:1.0:1769074799.519494:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625197440:12345-0@lo:41:osp-pre-0-1.0 Request processed in 587us (2444us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074799.519504:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2664 00000100:00100000:0.0:1769074799.520232:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625197440:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074799.520444:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074799.520456:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625197568:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074799.520505:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074799.520541:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197568 00000100:00100000:0.0:1769074799.520547:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197568 00000100:00100000:0.0:1769074799.520583:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2665 00000100:00100000:0.0:1769074799.520591:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625197568:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074799.520619:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:0.0:1769074799.520660:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625197568:12345-0@lo:41:osp-pre-1-0.0 Request processed in 69us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074799.520665:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2665 00000100:00100000:0.0:1769074799.520697:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625197568:0@lo:41:osp-pre-1-0.0 00000004:00080000:1.0:1769074800.029004:0:32395:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1787871194376ns, next at 1788871191006ns, rc = 0 00000004:00080000:1.0:1769074800.029059:0:32397:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1787871252007ns, next at 1788871251280ns, rc = 0 00000100:00100000:1.0:1769074800.029098:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625197696:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074800.029166:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074800.029518:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625197824:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074800.029547:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074800.029591:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197696 00000100:00100000:1.0:1769074800.029598:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197696 00000100:00100000:1.0:1769074800.029649:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197824 00000100:00100000:1.0:1769074800.029653:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197824 00000100:00100000:1.0:1769074800.029693:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1188 00000100:00100000:1.0:1769074800.029703:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625197696:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074800.029727:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 495457600 pending 2121728 free 3905998848 avail 3674476544 00000100:00100000:1.0:1769074800.029781:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625197696:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074800.029789:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1188 00000100:00100000:1.0:1769074800.029840:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1189 00000100:00100000:1.0:1769074800.029847:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625197824:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074800.029867:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3900067840 avail 3668545536 00000100:00100000:1.0:1769074800.029901:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625197824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074800.029980:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1189 00000100:00080000:1.0:1769074800.030007:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074800.030011:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074800.030073:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625197824:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074800.030112:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625197696:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074800.030440:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625197952:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074800.030473:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1769074800.030558:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074800.030560:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625197952 00000100:00100000:0.0:1769074800.030568:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625197952 00000100:00100000:0.0:1769074800.030666:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1190 00000100:00100000:0.0:1769074800.030675:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625197952:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074800.030689:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 495457600 pending 2121728 free 3905998848 avail 3674476544 00000100:00100000:0.0:1769074800.030733:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625197952:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074800.030741:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1190 00000100:00080000:2.0:1769074800.030786:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074800.030931:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625197952:0@lo:13:osp-pre-0-1.0 00000004:00080000:1.0:1769074800.034442:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1960 committed 00000004:00080000:1.0:1769074800.034471:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1956 committed 00000001:00080000:1.0:1769074800.034501:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944026 is committed 00000001:00100000:1.0:1769074800.034515:0:32377:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000004:00080000:1.0:1769074800.034520:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1955 committed 00000004:00080000:1.0:1769074800.034523:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1959 committed 00000100:00100000:1.0:1769074800.034686:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198080:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1769074800.034753:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074800.035209:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198592:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1769074800.035240:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074800.035262:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074800.035268:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198208:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1769074800.035290:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074800.035321:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198080 00000100:00100000:1.0:1769074800.035328:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198080 00000100:00100000:1.0:1769074800.035369:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198592 00000100:00100000:1.0:1769074800.035373:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198592 00000100:00100000:1.0:1769074800.035397:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198208 00000100:00100000:1.0:1769074800.035399:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198208 00000100:00100000:1.0:1769074800.035419:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14410 00000100:00100000:1.0:1769074800.035426:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625198080:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1769074800.035464:0:28954:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:700 count 1 00002000:00010000:1.0:1769074800.035498:0:28954:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bc:0x0] initial LVB size: 1048576, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x7e8 00000100:00100000:3.0:1769074800.035504:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14411 00010000:00010000:1.0:1769074800.035506:0:28954:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074800.035517:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625198592:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1769074800.035522:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.035526:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.035532:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00002000:00080000:3.0:1769074800.035534:0:27844:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:700 count 1 00010000:00010000:1.0:1769074800.035550:0:28954:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074800.035558:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bc:0x0] initial LVB size: 4194304, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x1008 00010000:00010000:1.0:1769074800.035560:0:28954:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.035569:0:27844:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.035571:0:28954:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.035592:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074800.035595:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074800.035598:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.035603:0:28954:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074800.035613:0:27844:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.035622:0:27844:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.035630:0:27844:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.035866:0:27844:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1769074800.036294:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036304:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036310:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036315:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67b446e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074800.036319:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036326:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036340:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2ffdc7bb (ffff89b681690300) 00010000:00010000:2.0:1769074800.036343:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074800.036347:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036353:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67b446e00/0x7fe2323adfd54743 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x2ffdc7bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074800.036374:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074800.036381:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074800.036386:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074800.036391:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67fa3d600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074800.036394:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074800.036400:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074800.036408:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b681690f00) 00010000:00010000:2.0:1769074800.036410:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074800.036413:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074800.036419:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67fa3d600/0x7fe2323adfd547dd lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074800.036448:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14412 00000100:00100000:2.0:1769074800.036456:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625198208:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1769074800.036471:0:8399:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:702 count 1 00002000:00010000:2.0:1769074800.036491:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2be:0x0] initial LVB size: 2097152, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x808 00010000:00010000:2.0:1769074800.036497:0:8399:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.036506:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.036508:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.036512:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.036522:0:8399:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.036530:0:8399:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.036538:0:8399:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.036554:0:8399:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1769074800.037356:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1769074800.037364:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037374:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074800.037377:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037398:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037414:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037421:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867e00/0x7fe2323adfd5481c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037438:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1769074800.037443:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037449:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074800.037452:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037459:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037468:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037475:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd54823 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037494:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1769074800.037498:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037504:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074800.037506:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037513:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037522:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037528:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd5482a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.037541:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074800.037542:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074800.037544:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074800.039647:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074800.040511:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625198080:12345-0@lo:6:osp-syn-1-0.0 Request processed in 5077us (5749us total) trans 17179874597 rc 0/0 00000100:00100000:1.0:1769074800.040524:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14410 00000100:00100000:2.0:1769074800.040968:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625198208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 4512us (5675us total) trans 17179874598 rc 0/0 00000100:00100000:2.0:1769074800.040977:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14412 00010000:00010000:2.0:1769074800.040998:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074800.041016:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199104:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1769074800.041054:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074800.041088:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1769074800.041113:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1769074800.041208:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5461e7800/2, rc 0, transno 17179874598 00000100:00100000:2.0:1769074800.041222:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198208:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1769074800.041239:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5461e5c00/2, rc 0, transno 17179874597 00000100:00100000:2.0:1769074800.041242:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198080:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1769074800.041256:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074800.041262:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198720:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1769074800.041286:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074800.041308:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199232:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1769074800.041329:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074800.041347:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199744:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1769074800.041367:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074800.041402:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199104 00000100:00100000:2.0:1769074800.041410:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199104 00000100:00100000:2.0:1769074800.041447:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199616 00000100:00100000:2.0:1769074800.041450:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199616 00000100:00100000:2.0:1769074800.041468:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198720 00000100:00100000:2.0:1769074800.041471:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198720 00000100:00100000:2.0:1769074800.041487:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199232 00000100:00100000:2.0:1769074800.041490:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199232 00000100:00100000:2.0:1769074800.041509:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199744 00000100:00100000:2.0:1769074800.041512:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199744 00000100:00100000:2.0:1769074800.041532:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14413 00000100:00100000:2.0:1769074800.041541:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625199104:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1769074800.041568:0:8396:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:706 count 1 00002000:00010000:2.0:1769074800.041599:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c2:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x0 00010000:00010000:2.0:1769074800.041607:0:8396:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041621:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.041624:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.041630:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041643:0:8396:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041653:0:8396:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041662:0:8396:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041679:0:8396:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1769074800.041887:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+10:3649:x1855007625199104:12345-0@lo:6:osp-syn-1-0.0 Request processed in 347us (832us total) trans 17179874599 rc 0/0 00000100:00100000:2.0:1769074800.041896:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14413 00000100:00100000:2.0:1769074800.041916:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14414 00000100:00100000:2.0:1769074800.041923:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625199616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1769074800.041937:0:8396:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:701 count 1 00002000:00010000:2.0:1769074800.041954:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bd:0x0] initial LVB size: 3145728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x1000 00010000:00010000:2.0:1769074800.041960:0:8396:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041969:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.041971:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.041974:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041985:0:8396:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.041994:0:8396:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.042002:0:8396:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.042017:0:8396:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1769074800.042263:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14415 00000100:00100000:1.0:1769074800.042273:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625198720:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1769074800.042287:0:8399:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:703 count 1 00002000:00010000:1.0:1769074800.042304:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bf:0x0] initial LVB size: 4805728, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x14b0 00010000:00010000:1.0:1769074800.042310:0:8399:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.042318:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.042320:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.042323:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.042333:0:8399:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.042340:0:8399:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.042347:0:8399:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.042362:0:8399:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1769074800.044054:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625198592:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8534us (8805us total) trans 21474841266 rc 0/0 00000100:00100000:3.0:1769074800.044067:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14411 00000100:00100000:3.0:1769074800.044098:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14416 00000100:00100000:3.0:1769074800.044110:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+10:3649:x1855007625199232:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1769074800.044133:0:27844:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:701 count 1 00002000:00010000:3.0:1769074800.044160:0:27844:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2bd:0x0] initial LVB size: 4194304, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0x1fe8 00010000:00010000:3.0:1769074800.044169:0:27844:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.044184:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074800.044187:0:27844:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074800.044270:0:27844:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.044284:0:27844:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.044295:0:27844:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.044302:0:27844:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.044314:0:27844:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1769074800.049425:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198464:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1769074800.049500:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049529:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198976:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1769074800.049553:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049578:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625199488:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1769074800.049598:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049614:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625200000:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1769074800.049632:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049651:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:0.0:1769074800.049657:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198336:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1769074800.049677:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049687:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198848:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1769074800.049708:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049718:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625199360:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1769074800.049734:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.049743:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625199872:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1769074800.049759:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074800.052892:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198464 00000100:00100000:0.0:1769074800.052908:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198464 00000100:00100000:0.0:1769074800.052961:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14417 00000100:00100000:0.0:1769074800.052972:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625199744:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1769074800.053005:0:9048:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:702 count 1 00002000:00010000:0.0:1769074800.053041:0:9048:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2be:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:0.0:1769074800.053049:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.053064:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.053065:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.053071:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.053082:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.053092:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.053100:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.053111:0:9048:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1769074800.059400:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198976 00000100:00100000:2.0:1769074800.059416:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198976 00000100:00100000:2.0:1769074800.059465:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199488 00000100:00100000:2.0:1769074800.059469:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199488 00000100:00100000:2.0:1769074800.059481:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200000 00000100:00100000:2.0:1769074800.059484:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200000 00000100:00100000:2.0:1769074800.059492:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198336 00000100:00100000:2.0:1769074800.059494:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198336 00000100:00100000:2.0:1769074800.059503:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625198848 00000100:00100000:2.0:1769074800.059506:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625198848 00000100:00100000:2.0:1769074800.059515:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199360 00000100:00100000:2.0:1769074800.059517:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199360 00000100:00100000:2.0:1769074800.059525:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625199872 00000100:00100000:2.0:1769074800.059527:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625199872 00000100:00100000:2.0:1769074800.059547:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14418 00000100:00100000:2.0:1769074800.059556:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625198464:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1769074800.059588:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:699 count 1 00002000:00010000:2.0:1769074800.059622:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bb:0x0] initial LVB size: 2097152, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:2.0:1769074800.059630:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.059651:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.059653:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.059658:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.059673:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.059683:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.059692:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.059705:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:2.0:1769074800.060843:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625198464:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1283us (11338us total) trans 21474841269 rc 0/0 00000100:00100000:2.0:1769074800.060864:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14418 00000100:00100000:2.0:1769074800.060890:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14419 00000100:00100000:2.0:1769074800.060899:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625198976:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1769074800.060918:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:705 count 1 00002000:00010000:2.0:1769074800.060940:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c1:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:2.0:1769074800.060945:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.060957:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.060959:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.060963:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.060974:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.060984:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.060992:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.061002:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/7 locks to regular blp list, 7 blwis in pool 00000100:00100000:0.0:1769074800.061360:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14420 00000100:00100000:0.0:1769074800.061372:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625199488:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1769074800.061387:0:8398:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:697 count 1 00002000:00010000:0.0:1769074800.061404:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2b9:0x0] initial LVB size: 2097152, mtime: 0x6971f05b, atime: 0x0, ctime: 0x6971f05b, blocks: 0xfe8 00010000:00010000:0.0:1769074800.061410:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.061419:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.061421:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.061424:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.061435:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.061443:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.061450:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.061459:0:8398:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/8 locks to regular blp list, 8 blwis in pool 00000100:00100000:0.0:1769074800.063118:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625199488:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1743us (13511us total) trans 21474841270 rc 0/0 00000100:00100000:0.0:1769074800.063130:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14420 00000100:00100000:0.0:1769074800.063157:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14421 00000100:00100000:0.0:1769074800.063167:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625200000:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1769074800.063187:0:8398:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:704 count 1 00002000:00010000:0.0:1769074800.063213:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c0:0x0] initial LVB size: 4194304, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x0 00010000:00010000:0.0:1769074800.063220:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063232:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.063234:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.063239:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063250:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063258:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063265:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063275:0:8398:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/9 locks to regular blp list, 9 blwis in pool 00000100:00100000:0.0:1769074800.063393:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625200000:12345-0@lo:6:osp-syn-0-0.0 Request processed in 229us (13759us total) trans 21474841271 rc 0/0 00000100:00100000:0.0:1769074800.063400:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14421 00000100:00100000:0.0:1769074800.063413:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14422 00000100:00100000:0.0:1769074800.063419:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625198336:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1769074800.063430:0:8398:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:698 count 1 00002000:00010000:0.0:1769074800.063445:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2ba:0x0] initial LVB size: 2097152, mtime: 0x6971f05c, atime: 0x6971f05c, ctime: 0x6971f063, blocks: 0x800 00010000:00010000:0.0:1769074800.063450:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063458:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.063459:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.063462:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063470:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063477:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063483:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063491:0:8398:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/10 locks to regular blp list, 10 blwis in pool 00000100:00100000:0.0:1769074800.063912:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14423 00000100:00100000:0.0:1769074800.063922:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625198848:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1769074800.063940:0:28519:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:704 count 1 00002000:00010000:0.0:1769074800.063958:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c0:0x0] initial LVB size: 3145728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x1000 00010000:00010000:0.0:1769074800.063964:0:28519:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063972:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.063974:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074800.063978:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063988:0:28519:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.063997:0:28519:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.064005:0:28519:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074800.064013:0:28519:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/11 locks to regular blp list, 11 blwis in pool 00000100:00100000:2.0:1769074800.067798:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+16:3652:x1855007625198976:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6890us (18233us total) trans 17179874602 rc 0/0 00000100:00100000:2.0:1769074800.067836:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14419 00000100:00100000:2.0:1769074800.069278:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14424 00000100:00100000:2.0:1769074800.069298:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625199360:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1769074800.069330:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:707 count 1 00002000:00010000:2.0:1769074800.069361:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c3:0x0] initial LVB size: 8388608, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x2000 00010000:00010000:2.0:1769074800.069370:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.069385:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.069388:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074800.069393:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.069412:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.069421:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.069429:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.069442:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/12 locks to regular blp list, 12 blwis in pool 00000100:00100000:1.0:1769074800.071884:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+16:3649:x1855007625198720:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29605us (30588us total) trans 17179874600 rc 0/0 00000100:00100000:1.0:1769074800.071898:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14415 00000100:00100000:1.0:1769074800.071930:0:8399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14425 00000100:00100000:1.0:1769074800.071940:0:8399:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625199872:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1769074800.071966:0:8399:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:703 count 1 00002000:00010000:1.0:1769074800.071995:0:8399:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2bf:0x0] initial LVB size: 4805728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x4b0 00010000:00010000:1.0:1769074800.072004:0:8399:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.072021:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.072023:0:8399:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.072028:0:8399:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.072041:0:8399:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.072050:0:8399:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.072058:0:8399:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.072070:0:8399:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/13 locks to regular blp list, 13 blwis in pool 00000100:00100000:1.0:1769074800.072684:0:8399:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+16:3652:x1855007625199872:12345-0@lo:6:osp-syn-0-0.0 Request processed in 745us (22921us total) trans 21474841273 rc 0/0 00000100:00100000:1.0:1769074800.072692:0:8399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14425 00000100:00100000:1.0:1769074800.077326:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074800.077355:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625200128:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1769074800.077434:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074800.077482:0:28954:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200128 00000100:00100000:1.0:1769074800.077490:0:28954:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200128 00000100:00100000:1.0:1769074800.077532:0:28954:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14426 00000100:00100000:1.0:1769074800.077540:0:28954:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+16:3650:x1855007625200128:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1769074800.077570:0:28954:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:708 count 1 00002000:00010000:1.0:1769074800.077602:0:28954:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c4:0x0] initial LVB size: 4805728, mtime: 0x6971f063, atime: 0x0, ctime: 0x6971f063, blocks: 0x4b0 00010000:00010000:1.0:1769074800.077610:0:28954:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.077624:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.077626:0:28954:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074800.077631:0:28954:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.077644:0:28954:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.077659:0:28954:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.077667:0:28954:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074800.077678:0:28954:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/14 locks to regular blp list, 14 blwis in pool 00000100:00100000:1.0:1769074800.081658:0:28954:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+17:3650:x1855007625200128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 4108us (4213us total) trans 17179874605 rc 0/0 00000100:00100000:1.0:1769074800.081673:0:28954:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14426 00000004:00080000:1.0:1769074800.081752:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d0e4a80/2, rc 0, transno 17179874605 00000100:00100000:1.0:1769074800.081759:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625200128:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1769074800.082370:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625198336:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18949us (32690us total) trans 21474841272 rc 0/0 00000100:00100000:1.0:1769074800.082378:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14422 00000100:00100000:1.0:1769074800.083186:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625199744:12345-0@lo:6:osp-syn-0-0.0 Request processed in 30214us (41815us total) trans 21474841268 rc 0/0 00000100:00100000:1.0:1769074800.083198:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14417 00000100:00100000:3.0:1769074800.085047:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+16:3649:x1855007625199232:12345-0@lo:6:osp-syn-1-0.0 Request processed in 40933us (43708us total) trans 17179874601 rc 0/0 00000100:00100000:3.0:1769074800.085059:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14416 00000100:00100000:0.0:1769074800.086026:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+15:3652:x1855007625198848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22101us (36313us total) trans 17179874603 rc 0/0 00000100:00100000:0.0:1769074800.086037:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14423 00010000:00010000:2.0:1769074800.090881:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 13 total in blp. (13 blwis in pool) 00010000:00010000:2.0:1769074800.090898:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090911:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.090915:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090926:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090947:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090953:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88a00/0x7fe2323adfd54831 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090976:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 12 total in blp. (12 blwis in pool) 00010000:00010000:2.0:1769074800.090980:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090986:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.090988:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.090994:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091003:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091009:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54838 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091020:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 11 total in blp. (11 blwis in pool) 00010000:00010000:2.0:1769074800.091025:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091030:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091032:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091039:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091046:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091052:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866800/0x7fe2323adfd5483f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091064:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 10 total in blp. (10 blwis in pool) 00010000:00010000:2.0:1769074800.091068:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091074:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091077:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091089:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091097:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091103:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba400/0x7fe2323adfd54846 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27844 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091115:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 9 total in blp. (9 blwis in pool) 00010000:00010000:2.0:1769074800.091119:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091128:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602ae00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091131:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091137:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091146:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091153:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd5484d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091165:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 8 total in blp. (8 blwis in pool) 00010000:00010000:2.0:1769074800.091169:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091175:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091177:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091183:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091192:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091203:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54854 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091215:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 7 total in blp. (7 blwis in pool) 00010000:00010000:2.0:1769074800.091218:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091223:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091226:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091234:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091243:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091250:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89000/0x7fe2323adfd5485b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091261:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 6 total in blp. (6 blwis in pool) 00010000:00010000:2.0:1769074800.091265:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091270:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602a600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091273:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091279:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091287:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091293:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602a600/0x7fe2323adfd54862 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091304:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:2.0:1769074800.091307:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091313:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602a000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091315:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091322:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091329:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091336:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602a000/0x7fe2323adfd54869 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091350:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1769074800.091353:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091358:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602a200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091361:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091368:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091375:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091381:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54870 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091390:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1769074800.091393:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091399:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602b000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091402:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091408:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091415:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091421:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67602b000/0x7fe2323adfd54877 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091429:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1769074800.091433:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091438:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091441:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091447:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091456:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091462:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88200/0x7fe2323adfd5487e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091474:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1769074800.091477:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091487:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091489:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091496:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091505:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091511:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867400/0x7fe2323adfd54885 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8399 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091522:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074800.091526:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091531:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074800.091534:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091540:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091548:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091557:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867800/0x7fe2323adfd5488c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28954 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.091568:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074800.091570:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074800.091571:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074800.091584:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074800.091611:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1769074800.091667:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759d880/2, rc 0, transno 17179874600 00000100:00100000:2.0:1769074800.091679:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198720:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1769074800.091691:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b545109f80/2, rc 0, transno 17179874601 00000100:00100000:2.0:1769074800.091694:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199232:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1769074800.091705:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5444a7100/2, rc 0, transno 21474841268 00000100:00100000:2.0:1769074800.091709:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199744:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074800.091718:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759d500/2, rc 0, transno 17179874599 00000100:00100000:2.0:1769074800.091721:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199104:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1769074800.091732:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d103100/2, rc 0, transno 21474841266 00000100:00100000:2.0:1769074800.091735:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d103100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625198592:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074800.091763:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ef800/2, rc 0, transno 21474841272 00000100:00100000:2.0:1769074800.091766:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198336:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074800.091775:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759fb80/2, rc 0, transno 17179874603 00000100:00100000:2.0:1769074800.091778:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198848:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1769074800.091788:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5444a4a80/2, rc 0, transno 21474841273 00000100:00100000:2.0:1769074800.091791:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625199872:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074800.091799:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ed880/2, rc 0, transno 21474841269 00000100:00100000:2.0:1769074800.091802:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198464:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074800.091838:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759c000/2, rc 0, transno 17179874602 00000100:00100000:2.0:1769074800.091842:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625198976:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1769074800.091862:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5444a4700/2, rc 0, transno 21474841270 00000100:00100000:2.0:1769074800.091865:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625199488:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074800.091875:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5444a4000/2, rc 0, transno 21474841271 00000100:00100000:2.0:1769074800.091878:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625200000:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1769074800.092048:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+14:3652:x1855007625199360:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22759us (42311us total) trans 17179874604 rc 0/0 00000100:00100000:2.0:1769074800.092058:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14424 00000004:00080000:0.0:1769074800.092156:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547598000/2, rc 0, transno 17179874604 00000100:00100000:0.0:1769074800.092163:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625199360:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1769074800.092568:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625199616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 50645us (51452us total) trans 21474841267 rc 0/0 00000100:00100000:2.0:1769074800.092577:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14414 00000004:00080000:0.0:1769074800.092599:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5444a6680/2, rc 0, transno 21474841267 00000100:00100000:0.0:1769074800.092606:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625199616:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1769074800.374561:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074800.374601:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616962048 00000100:00100000:3.0:1769074800.374611:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616962048 00000100:00100000:3.0:1769074800.374669:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13394 00000100:00100000:3.0:1769074800.374678:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:63717:x1855007616962048:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074800.374702:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074800.374835:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54893 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c3fc expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.374915:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5489a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074800.374932:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5489a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074800.374944:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5489a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074800.374993:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1769074800.374997:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074800.375021:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5489a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074800.375046:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54893 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3fc expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.375079:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5489a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3fc expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074800.375087:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074800.375090:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074800.375094:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:3.0:1769074800.375166:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:63717:x1855007616962048:12345-192.168.204.60@tcp:101:dd.0 Request processed in 488us (614us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074800.375174:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13394 00000100:00080000:3.0:1769074800.375213:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074800.383767:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074800.383934:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616962304 00000100:00100000:2.0:1769074800.383944:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616962304 00000100:00100000:2.0:1769074800.387219:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13395 00000100:00100000:2.0:1769074800.387244:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:63717:x1855007616962304:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074800.387271:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074800.387274:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074800.387301:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd5489a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3fc expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.387319:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074800.387324:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074800.387339:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd548a1 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c403 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074800.387481:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074800.387497:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074800.387507:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074800.387521:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074800.387527:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074800.387533:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000001:00020000:2.0:1769074800.387587:0:6512:0:(mdt_open.c:1759:mdt_reint_open()) cfs_fail_timeout id 175 sleeping for 10000ms 00000100:00100000:1.0:1769074800.541555:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625200256:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074800.541642:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074800.541745:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200256 00000100:00100000:2.0:1769074800.541752:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200256 00000100:00080000:3.0:1769074800.541800:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074800.541833:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1191 00000100:00100000:2.0:1769074800.541852:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625200256:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074800.541881:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:2.0:1769074800.541928:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625200256:12345-0@lo:13:osp-pre-1-1.0 Request processed in 87us (286us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074800.541935:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1191 00000100:00100000:3.0:1769074800.541971:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625200256:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074801.053474:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074802.080349:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074802.589088:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625200640:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589161:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074802.589212:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625201152:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589234:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074802.589251:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625201664:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589278:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074802.589300:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1769074802.589305:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625200768:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589326:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074802.589346:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625201280:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589365:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074802.589379:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625201792:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589397:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074802.589427:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200640 00000100:00100000:1.0:1769074802.589434:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200640 00000100:00100000:1.0:1769074802.589475:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2365 00000100:00100000:1.0:1769074802.589483:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625200640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589535:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625200640:12345-0@lo:400:kworker.0 Request processed in 52us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074802.589542:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2365 00000100:00100000:1.0:1769074802.589559:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201152 00000100:00100000:1.0:1769074802.589563:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201152 00000100:00100000:1.0:1769074802.589584:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201280 00000100:00100000:1.0:1769074802.589587:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201280 00000100:00100000:1.0:1769074802.589602:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2366 00000100:00100000:1.0:1769074802.589608:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625201152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589636:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625201152:12345-0@lo:400:kworker.0 Request processed in 28us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074802.589642:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2366 00000100:00100000:1.0:1769074802.589653:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2367 00000100:00100000:1.0:1769074802.589658:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625201280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589685:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625201280:12345-0@lo:400:kworker.0 Request processed in 26us (318us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074802.589691:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2367 00000100:00080000:1.0:1769074802.589711:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074802.589714:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074802.589741:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625201280:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589752:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625200640:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.589759:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625201152:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.589918:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625200384:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.589953:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.589983:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625200896:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590006:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.590025:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625201408:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590046:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.590060:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625201920:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590081:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.590098:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1769074802.590104:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625200512:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590128:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.590147:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625201024:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590168:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.590177:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625201536:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590196:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074802.590222:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201664 00000100:00100000:0.0:1769074802.590227:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201664 00000100:00100000:0.0:1769074802.590262:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200768 00000100:00100000:0.0:1769074802.590265:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200768 00000100:00100000:0.0:1769074802.590282:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201792 00000100:00100000:0.0:1769074802.590284:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201792 00000100:00100000:0.0:1769074802.590299:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200896 00000100:00100000:0.0:1769074802.590302:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200896 00000100:00100000:0.0:1769074802.590318:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1480 00000100:00100000:0.0:1769074802.590325:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625201664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590358:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+13:3649:x1855007625201664:12345-0@lo:400:kworker.0 Request processed in 33us (1079us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074802.590367:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1480 00000100:00100000:0.0:1769074802.590380:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1481 00000100:00100000:0.0:1769074802.590385:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625200768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590411:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625200768:12345-0@lo:400:kworker.0 Request processed in 26us (1083us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074802.590417:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1481 00000100:00100000:0.0:1769074802.590427:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1482 00000100:00100000:0.0:1769074802.590432:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625201792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590456:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+14:3649:x1855007625201792:12345-0@lo:400:kworker.0 Request processed in 24us (1059us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074802.590462:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1482 00000100:00100000:0.0:1769074802.590472:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1483 00000100:00100000:0.0:1769074802.590478:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625200896:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590507:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625200896:12345-0@lo:400:kworker.0 Request processed in 30us (500us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074802.590513:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1483 00000100:00100000:0.0:1769074802.590531:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200512 00000100:00100000:0.0:1769074802.590534:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200512 00000100:00100000:0.0:1769074802.590556:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201536 00000100:00100000:0.0:1769074802.590559:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201536 00000100:00100000:0.0:1769074802.590575:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2666 00000100:00100000:0.0:1769074802.590580:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625200512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590613:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625200512:12345-0@lo:400:kworker.0 Request processed in 32us (483us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074802.590619:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2666 00000100:00100000:0.0:1769074802.590631:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2667 00000100:00100000:0.0:1769074802.590637:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625201536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590662:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625201536:12345-0@lo:400:kworker.0 Request processed in 26us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074802.590667:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2667 00000100:00100000:0.0:1769074802.590699:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625200512:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590708:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625201536:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.590716:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625200896:0@lo:400:kworker.0 00000100:00100000:2.0:1769074802.590998:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201408 00000100:00100000:2.0:1769074802.591003:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201408 00000100:00100000:2.0:1769074802.591029:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201920 00000100:00100000:2.0:1769074802.591031:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201920 00000100:00100000:2.0:1769074802.591042:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625201024 00000100:00100000:2.0:1769074802.591044:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625201024 00000100:00100000:2.0:1769074802.591059:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2368 00000100:00100000:2.0:1769074802.591064:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625201408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074802.591115:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625201408:12345-0@lo:400:kworker.0 Request processed in 50us (1067us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074802.591120:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2368 00000100:00100000:2.0:1769074802.591134:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2369 00000100:00100000:2.0:1769074802.591140:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625201920:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074802.591167:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625201920:12345-0@lo:400:kworker.0 Request processed in 27us (1084us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074802.591172:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2369 00000100:00100000:2.0:1769074802.591185:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2370 00000100:00100000:2.0:1769074802.591190:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625201024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074802.591215:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625201024:12345-0@lo:400:kworker.0 Request processed in 26us (1047us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074802.591221:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2370 00000100:00100000:0.0:1769074802.591315:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625201024:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.591326:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625201408:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.591335:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625201920:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.591418:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625200384 00000100:00100000:1.0:1769074802.591422:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625200384 00000100:00100000:1.0:1769074802.591451:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1769074802.591458:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681596c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625200384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.591565:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681596c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625200384:12345-0@lo:400:kworker.0 Request processed in 107us (1610us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074802.591573:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00080000:1.0:1769074802.591615:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074802.591619:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074802.591622:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074802.591625:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074802.591647:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625200768:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.591659:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625201792:0@lo:400:kworker.0 00000100:00100000:1.0:1769074802.591670:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625201664:0@lo:400:kworker.0 00000100:00100000:0.0:1769074802.601752:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625200384:0@lo:400:kworker.0 00000001:02000400:2.0:1769074802.987566:0:36827:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: mirror extend 00000100:00100000:1.0:1769074803.070401:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.083610:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616962432 00000100:00100000:2.0:1769074803.083626:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616962432 00000100:00100000:2.0:1769074803.085839:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13396 00000100:00100000:2.0:1769074803.085855:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:63851:x1855007616962432:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074803.085877:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074803.085899:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd548b6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c40a expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00000100:00080000:1.0:1769074803.086091:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1769074803.086222:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd548bd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.086238:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd548bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.086251:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd548bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074803.086295:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074803.086299:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074803.086317:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd548bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.086342:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd548b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c40a expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.086361:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd548bd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c40a expref: 6 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.086370:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074803.086373:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074803.086377:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074803.086600:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63851:x1855007616962432:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 748us (16207us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074803.086609:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13396 00000100:00100000:0.0:1769074803.099404:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.099713:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616962688 00000100:00100000:2.0:1769074803.099724:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616962688 00000100:00100000:2.0:1769074803.099780:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13397 00000100:00100000:2.0:1769074803.099790:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63851:x1855007616962688:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074803.099873:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074803.099893:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd548c4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c411 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.099967:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.100381:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.100393:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.100407:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.100414:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.104672:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.104779:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074803.105495:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd548d9 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105513:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd548d9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105523:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd548d9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074803.105535:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00010000:00010000:3.0:1769074803.105630:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105637:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105643:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105650:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105655:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074803.105659:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105666:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105679:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074803.105682:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074803.105686:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105692:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd548cb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105704:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105709:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105714:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105719:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074803.105721:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105727:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105738:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b64effa500) 00010000:00010000:3.0:1769074803.105740:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074803.105743:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105749:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd548d2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.105794:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd548c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c411 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.105839:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd548d9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c411 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.105846:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074803.105849:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074803.105851:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbc00, rc 0) 00000100:00100000:3.0:1769074803.105943:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:63851:x1855007616962688:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6153us (6548us total) trans 8589944027 rc 0/0 00000100:00100000:3.0:1769074803.105953:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13397 00000100:00100000:3.0:1769074803.114347:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074803.115596:0:6523:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616962816 00000100:00100000:0.0:1769074803.115615:0:6523:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616962816 00000100:00100000:0.0:1769074803.115698:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 27 00000100:00100000:0.0:1769074803.115708:0:6523:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63851:x1855007616962816:12345-192.168.204.60@tcp:900:lfs.0 00000100:00100000:0.0:1769074803.115830:0:6523:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63851:x1855007616962816:12345-192.168.204.60@tcp:900:lfs.0 Request processed in 122us (1492us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074803.115838:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 27 00000100:00080000:2.0:1769074803.116161:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074803.158189:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.158230:0:6521:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616962944 00000100:00100000:1.0:1769074803.158241:0:6521:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616962944 00000100:00100000:1.0:1769074803.158306:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 10 00000100:00100000:1.0:1769074803.158316:0:6521:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqm_0000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:63851:x1855007616962944:12345-192.168.204.60@tcp:700:lfs.0 00000100:00100000:1.0:1769074803.158410:0:6521:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqm_0000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:63851:x1855007616962944:12345-192.168.204.60@tcp:700:lfs.0 Request processed in 94us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.158417:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 10 00000100:00100000:0.0:1769074803.170161:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.170266:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616963200 00000100:00100000:3.0:1769074803.170278:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616963200 00000100:00100000:3.0:1769074803.170346:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13398 00000100:00100000:3.0:1769074803.170357:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:63851:x1855007616963200:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074803.170382:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074803.170385:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074803.170414:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd548bd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c40a expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.170433:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074803.170436:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074803.170448:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd548e7 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c418 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.173380:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 3/1,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074803.173398:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074803.173409:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074803.173420:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd1:0x1:0x0]: returning size 0 00010000:00010000:1.0:1769074803.173479:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd548e7 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.173500:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 3/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.173508:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b681697100) 00010000:00010000:1.0:1769074803.173511:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074803.173515:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:1.0:1769074803.174594:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:63851:x1855007616963200:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4231us (4436us total) trans 8589944028 rc 0/0 00000100:00100000:1.0:1769074803.174606:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13398 00000100:00100000:3.0:1769074803.190864:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074803.193551:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616963456 00000100:00100000:0.0:1769074803.193567:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616963456 00000100:00100000:0.0:1769074803.193622:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13399 00000100:00100000:0.0:1769074803.193632:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:63851:x1855007616963456:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074803.193656:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074803.193678:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b000/0x7fe2323adfd548fc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c41f expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.193792:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074803.193861:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.193870:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 4/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074803.193904:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 3/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.193917:0:10283:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 10 pid: 10283 timeout: 1891 lvb_type: 0 00000100:00100000:0.0:1769074803.193931:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625202048:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074803.194027:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499c5180 going to sleep for 11 seconds 00000100:00100000:1.0:1769074803.215929:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625202048:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074803.215961:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074803.220242:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.220344:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616963584 00000100:00100000:2.0:1769074803.220355:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616963584 00010000:00010000:2.0:1769074803.220398:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 11 pid: 10283 timeout: 1891 lvb_type: 0 00000100:00100000:2.0:1769074803.220443:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7178 00000100:00100000:2.0:1769074803.220453:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:2408:x1855007616963584:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074803.220469:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074803.220476:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 11 pid: 10283 timeout: 1891 lvb_type: 0 00010000:00010000:2.0:1769074803.220489:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 3/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 11 pid: 10283 timeout: 1891 lvb_type: 0 00010000:00010000:2.0:1769074803.220511:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd548ee lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c418 expref: 11 pid: 10283 timeout: 1891 lvb_type: 0 00010000:00010000:2.0:1769074803.220739:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b681697100) 00010000:00010000:2.0:1769074803.220744:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.220754:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.220761:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 4/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.221077:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074803.221832:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.221844:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074803.221866:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd1:0x1:0x0]: returning size 0 00000004:00080000:3.0:1769074803.222036:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c2:0x0], index 0: 0 00000004:00080000:3.0:1769074803.222045:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2c5:0x0], index 1: 0 00010000:00010000:3.0:1769074803.222167:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222176:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 2/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222182:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222188:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67602a200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074803.222192:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 3/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222199:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 4/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222214:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b681697100) 00010000:00010000:3.0:1769074803.222217:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074803.222221:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 2/0,0 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222227:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602a200/0x7fe2323adfd54903 lrc: 0/0,0 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.222280:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b000/0x7fe2323adfd548fc lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c41f expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.222290:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074803.222292:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074803.222296:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67602b000/0x7fe2323adfd548fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c41f expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.222304:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602b000, rc 0) 00000100:00100000:3.0:1769074803.222365:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616963456:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 28733us (31511us total) trans 8589944029 rc 301/301 00000100:00100000:3.0:1769074803.222375:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13399 00000100:00100000:2.0:1769074803.222721:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:2408:x1855007616963584:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 688us (907us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074803.222733:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7178 00000100:00100000:3.0:1769074803.227357:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.227522:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616963840 00000100:00100000:2.0:1769074803.227532:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616963840 00000100:00100000:0.0:1769074803.227647:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4705 00000100:00100000:0.0:1769074803.227658:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:63851:x1855007616963840:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074803.231867:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616963840:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 4196us (4507us total) trans 8589944030 rc 0/0 00000100:00100000:0.0:1769074803.231882:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4705 00000100:00100000:0.0:1769074803.234870:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.235197:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616963968 00000100:00100000:1.0:1769074803.235207:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616963968 00000100:00100000:1.0:1769074803.235327:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14427 00000100:00100000:1.0:1769074803.235339:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2409:x1855007616963968:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.235360:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.235379:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c426 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074803.235405:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c1:0x0] initial LVB size: 2097152, mtime: 0x6971f06e, atime: 0x0, ctime: 0x6971f06e, blocks: 0x1000 00010000:00010000:1.0:1769074803.235426:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.235428:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.235436:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 7 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.235460:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 7 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.235472:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866e00, rc 0) 00000100:00100000:1.0:1769074803.235531:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2409:x1855007616963968:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 195us (670us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.235538:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14427 00000100:00080000:2.0:1769074803.236364:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074803.251730:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.251842:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964096 00000100:00100000:3.0:1769074803.251852:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964096 00000100:00100000:3.0:1769074803.251909:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13400 00000100:00100000:3.0:1769074803.251918:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616964096:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074803.251943:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074803.251965:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54918 lrc: 2/0,0 mode: --/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c42d expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.252046:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd5491f lrc: 3/1,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.252059:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd5491f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.252071:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd5491f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074803.252117:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd1:0x1:0x0]: returning size 0 00000004:00200000:3.0:1769074803.252122:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074803.252142:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd5491f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074803.252178:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54918 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c42d expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.252194:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd5491f lrc: 3/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c42d expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.252203:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:3.0:1769074803.252205:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1769074803.252209:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb800, rc 0) 00000100:00100000:3.0:1769074803.252266:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:63851:x1855007616964096:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 348us (547us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.252283:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13400 00000100:00100000:3.0:1769074803.257668:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.257780:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.258446:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964352 00000100:00100000:1.0:1769074803.258459:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964352 00000100:00100000:0.0:1769074803.259551:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964224 00000100:00100000:0.0:1769074803.259562:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964224 00000100:00100000:0.0:1769074803.259640:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14428 00000100:00100000:0.0:1769074803.259651:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2409:x1855007616964224:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074803.259668:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074803.259692:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c434 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074803.259715:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c2:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1769074803.259733:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074803.259735:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074803.259741:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 8 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074803.259754:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 8 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074803.259764:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67602ae00, rc 0) 00000100:00100000:0.0:1769074803.259861:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2409:x1855007616964224:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 211us (2080us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074803.259869:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14428 00000100:00100000:1.0:1769074803.260927:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14429 00000100:00100000:1.0:1769074803.260940:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2409:x1855007616964352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.260954:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.260975:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c43b expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074803.260997:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c5:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074803.261014:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.261016:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.261034:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 7 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.261054:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 7 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.261069:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867000, rc 0) 00000100:00100000:1.0:1769074803.261134:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2409:x1855007616964352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 197us (3477us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.261140:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14429 00000100:00080000:2.0:1769074803.261194:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074803.267000:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.267361:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964608 00000100:00100000:1.0:1769074803.267370:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964608 00000100:00100000:1.0:1769074803.267481:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13401 00000100:00100000:1.0:1769074803.267493:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:63851:x1855007616964608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.267526:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.267552:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54934 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c442 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.267688:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd5493b lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.267703:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd5493b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.267716:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd5493b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:1.0:1769074803.267729:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00010000:00010000:1.0:1769074803.267874:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866a00/0x7fe2323adfd54934 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c442 expref: 14 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.267891:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd5493b lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c442 expref: 13 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.267905:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074803.267908:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074803.267911:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866400, rc 0) 00000100:00100000:1.0:1769074803.267975:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+13:63851:x1855007616964608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 484us (984us total) trans 8589944031 rc 0/0 00000100:00100000:1.0:1769074803.267982:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13401 00000100:00100000:0.0:1769074803.273306:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.273395:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964736 00000100:00100000:3.0:1769074803.273405:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964736 00000100:00100000:3.0:1769074803.273460:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14430 00000100:00100000:3.0:1769074803.273470:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2409:x1855007616964736:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074803.273529:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273545:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074803.273548:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074803.273552:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273566:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273592:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273599:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273606:0:9048:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273622:0:9048:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074803.273627:0:9048:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273635:0:9048:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273649:0:9048:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.273655:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54949 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074803.273735:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2409:x1855007616964736:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 265us (439us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.273743:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14430 00000100:00100000:0.0:1769074803.324369:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.324625:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964864 00000100:00100000:1.0:1769074803.324638:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964864 00000100:00100000:1.0:1769074803.324702:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14431 00000100:00100000:1.0:1769074803.324712:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2407:x1855007616964864:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074803.325400:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2407:x1855007616964864:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 685us (1037us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074803.325410:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14431 00000100:00100000:2.0:1769074803.343182:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.343483:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616964992 00000100:00100000:1.0:1769074803.343494:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616964992 00000100:00100000:1.0:1769074803.343580:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8737 00000100:00100000:1.0:1769074803.343590:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616964992:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074803.343627:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.343642:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.343644:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.343650:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.343663:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392050:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392248:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392258:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392266:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074803.392270:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392279:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392299:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.392305:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54950 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074803.392395:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616964992:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 48801us (49227us total) trans 0 rc 2097152/2097152 00000100:00100000:1.0:1769074803.392405:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8737 00000100:00100000:0.0:1769074803.404992:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.405113:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965120 00000100:00100000:1.0:1769074803.405125:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965120 00000100:00100000:1.0:1769074803.405191:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13402 00000100:00100000:1.0:1769074803.405200:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+13:63851:x1855007616965120:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.405223:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.405245:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54957 lrc: 2/0,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c449 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.405280:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 3/1,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.405294:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.405305:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.405418:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54957 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 15 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.405434:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 3/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 14 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.405442:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:1.0:1769074803.405445:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1769074803.405448:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866c00, rc 0) 00000100:00100000:1.0:1769074803.405515:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+14:63851:x1855007616965120:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 312us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.405523:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13402 00000100:00100000:0.0:1769074803.414400:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.414702:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965248 00000100:00100000:1.0:1769074803.414714:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965248 00000100:00100000:1.0:1769074803.414890:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8738 00000100:00100000:1.0:1769074803.414899:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616965248:12345-192.168.204.60@tcp:4:lfs.0 00010000:00010000:1.0:1769074803.414936:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.414961:0:8404:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 8 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.414969:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.414971:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.414975:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.415010:0:8404:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 8 pid: 8398 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.415035:0:8404:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 8 pid: 8398 timeout: 1891 lvb_type: 1 00000100:00100000:1.0:1769074803.415050:0:8404:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8404:1855007625202176:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074803.415137:0:8404:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074803.420839:0:8404:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8404:1855007625202176:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074803.420871:0:8404:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074803.421153:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.421418:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965376 00000100:00100000:2.0:1769074803.421426:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965376 00010000:00010000:2.0:1769074803.421463:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 9 pid: 8398 timeout: 1891 lvb_type: 1 00000100:00100000:2.0:1769074803.421503:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7179 00000100:00100000:2.0:1769074803.421523:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:2408:x1855007616965376:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074803.421537:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074803.421559:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 9 pid: 8398 timeout: 1891 lvb_type: 1 00010000:00010000:2.0:1769074803.421572:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 9 pid: 8398 timeout: 1891 lvb_type: 1 00010000:00010000:2.0:1769074803.421595:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67602ae00/0x7fe2323adfd54926 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c434 expref: 9 pid: 8398 timeout: 1891 lvb_type: 1 00010000:00010000:2.0:1769074803.421611:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074803.421613:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074803.421619:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.421628:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.421660:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074803.421721:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616965376:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 200us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074803.421732:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7179 00010000:00010000:1.0:1769074803.422846:0:8404:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.422859:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074803.459581:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841273 is committed 00000001:00080000:1.0:1769074803.459607:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841265 is committed 00000001:00100000:1.0:1769074803.462304:0:24714:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000001:00080000:1.0:1769074803.467350:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841274 is committed 00002000:00010000:0.0:1769074803.467579:0:8404:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:0.0:1769074803.467584:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb mtime from disk: 0 -> 1769074803 00002000:00010000:0.0:1769074803.467587:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb ctime from disk: 0 -> 1769074803 00002000:00010000:0.0:1769074803.467589:0:8404:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb blocks from disk: 0 -> 4096 00010000:00010000:0.0:1769074803.467604:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.467884:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.467893:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.467901:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074803.467904:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.467912:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.467929:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.467936:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54965 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074803.468109:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616965248:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 53206us (53714us total) trans 21474841274 rc 0/0 00000100:00100000:0.0:1769074803.468121:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8738 00000100:00100000:2.0:1769074803.478652:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.479114:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965504 00000100:00100000:1.0:1769074803.479125:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965504 00000100:00100000:1.0:1769074803.479302:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13403 00000100:00100000:1.0:1769074803.479312:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+14:63851:x1855007616965504:12345-192.168.204.60@tcp:44:lfs.0 00000001:00080000:1.0:1769074803.481553:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944031 is committed 00000100:00100000:2.0:1769074803.482130:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:63851:x1855007616965504:12345-192.168.204.60@tcp:44:lfs.0 Request processed in 2812us (3480us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074803.482150:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13403 00000100:00100000:2.0:1769074803.486680:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.487015:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074803.487388:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965632 00000100:00100000:0.0:1769074803.487399:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965632 00000100:00100000:0.0:1769074803.487501:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965760 00000100:00100000:0.0:1769074803.487505:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965760 00000100:00100000:3.0:1769074803.487543:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14432 00000100:00100000:3.0:1769074803.487557:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2409:x1855007616965632:12345-192.168.204.60@tcp:16:lfs.0 00000100:00100000:3.0:1769074803.487673:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2409:x1855007616965632:12345-192.168.204.60@tcp:16:lfs.0 Request processed in 118us (1003us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.487681:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14432 00000100:00100000:3.0:1769074803.487711:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14433 00000100:00100000:3.0:1769074803.487718:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2408:x1855007616965760:12345-192.168.204.60@tcp:16:lfs.0 00000100:00100000:3.0:1769074803.489326:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2408:x1855007616965760:12345-192.168.204.60@tcp:16:lfs.0 Request processed in 1599us (2303us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.489338:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14433 00000100:00100000:2.0:1769074803.504993:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.505104:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616965888 00000100:00100000:1.0:1769074803.505114:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616965888 00000100:00100000:1.0:1769074803.505183:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14434 00000100:00100000:1.0:1769074803.505194:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616965888:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074803.505241:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505256:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.505258:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.505262:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505276:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505303:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505317:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505325:0:9048:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505332:0:9048:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074803.505336:0:9048:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505344:0:9048:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505359:0:9048:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.505366:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866200/0x7fe2323adfd5496c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074803.505449:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616965888:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 248us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.505456:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14434 00000100:00100000:2.0:1769074803.509673:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.509784:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966144 00000100:00100000:1.0:1769074803.509803:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966144 00000100:00100000:1.0:1769074803.509902:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13404 00000100:00100000:1.0:1769074803.509912:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:63851:x1855007616966144:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:1.0:1769074803.509967:0:8855:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074803.509991:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd5491f lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c42d expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.510011:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:1.0:1769074803.510016:0:8855:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074803.510047:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,1 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.510060:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.510068:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00010000:1.0:1769074803.510202:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000a672b399 reply state 000000003cd7f924 transno 8589944032 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.510214:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.510220:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:1.0:1769074803.510223:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1769074803.510225:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074803.510328:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616966144:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 416us (663us total) trans 8589944032 rc 0/0 00000100:00100000:1.0:1769074803.510336:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13404 00000100:00100000:2.0:1769074803.516578:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.516674:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966272 00000100:00100000:3.0:1769074803.516686:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966272 00000100:00100000:3.0:1769074803.516752:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13405 00000100:00100000:3.0:1769074803.516763:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616966272:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074803.516789:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074803.516861:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 2/0,0 mode: --/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c450 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.516920:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 3/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.516933:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 3/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.516950:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:3.0:1769074803.516954:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074803.516957:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febaa00, rc 0) 00000100:00100000:3.0:1769074803.517013:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:63851:x1855007616966272:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 252us (447us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.517019:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13405 00000100:00100000:2.0:1769074803.521268:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074803.521371:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966400 00000100:00100000:0.0:1769074803.521382:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966400 00000100:00100000:0.0:1769074803.521461:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14435 00000100:00100000:0.0:1769074803.521472:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:63851:x1855007616966400:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074803.521490:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074803.521511:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c457 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074803.521533:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c2:0x0] initial LVB size: 2097152, mtime: 0x6971f073, atime: 0x0, ctime: 0x6971f073, blocks: 0x1000 00010000:00010000:0.0:1769074803.521554:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074803.521557:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074803.521564:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 7 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.521580:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 7 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.521591:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165400, rc 0) 00000100:00100000:0.0:1769074803.521660:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:63851:x1855007616966400:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 189us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074803.521667:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14435 00000100:00100000:2.0:1769074803.525293:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.525416:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966528 00000100:00100000:1.0:1769074803.525426:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966528 00000100:00100000:1.0:1769074803.525485:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14436 00000100:00100000:1.0:1769074803.525495:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:63851:x1855007616966528:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.525515:0:9048:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.525535:0:9048:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c45e expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.525566:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 8 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.525573:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.525576:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.525580:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 8 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.526078:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 8 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.526095:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 8 pid: 28955 timeout: 1891 lvb_type: 1 00000100:00100000:1.0:1769074803.526110:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625202304:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074803.526354:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074803.530230:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074803.530383:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966656 00000100:00100000:0.0:1769074803.530394:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966656 00010000:00010000:0.0:1769074803.530433:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 9 pid: 28955 timeout: 1891 lvb_type: 1 00000100:00100000:0.0:1769074803.530485:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7180 00000100:00100000:0.0:1769074803.530495:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:2408:x1855007616966656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074803.530510:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074803.530530:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 9 pid: 28955 timeout: 1891 lvb_type: 1 00010000:00010000:0.0:1769074803.530543:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 9 pid: 28955 timeout: 1891 lvb_type: 1 00010000:00010000:0.0:1769074803.530569:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074803.530571:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074803.530577:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 9 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.530586:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 9 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074803.530635:0:6498:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 9 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074803.530658:0:6498:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625202432:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074803.530763:0:6498:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499c5b40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.533575:0:6498:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6498:1855007625202432:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074803.533600:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074803.533660:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:2408:x1855007616966656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3166us (3438us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074803.533670:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7180 00010000:00010000:1.0:1769074803.535346:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547867000/0x7fe2323adfd5492d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c43b expref: 8 pid: 28955 timeout: 1891 lvb_type: 1 00000100:00100000:1.0:1769074803.535366:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9048:1855007625202304:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074803.535394:0:9048:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 7 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.535413:0:9048:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547866200, rc 0) 00000100:00100000:1.0:1769074803.535518:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:63851:x1855007616966528:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 10021us (10233us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.535527:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14436 00000100:00100000:3.0:1769074803.539156:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.539235:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966784 00000100:00100000:1.0:1769074803.539245:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966784 00000100:00100000:1.0:1769074803.539293:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14437 00000100:00100000:1.0:1769074803.539302:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2408:x1855007616966784:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:1.0:1769074803.539463:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074803.539467:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb mtime from disk: 1769074803 -> 1769074798 00002000:00010000:1.0:1769074803.539469:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb atime from disk: 0 -> 1769074798 00002000:00010000:1.0:1769074803.539471:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb ctime from disk: 1769074803 -> 1769074803 00002000:00010000:1.0:1769074803.539473:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074803.539540:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616966784:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 237us (749us total) trans 21474841275 rc 0/0 00000100:00100000:1.0:1769074803.539547:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14437 00000100:00100000:0.0:1769074803.543210:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.543307:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616966912 00000100:00100000:1.0:1769074803.543317:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616966912 00000100:00100000:1.0:1769074803.543372:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14438 00000100:00100000:1.0:1769074803.543382:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+7:2409:x1855007616966912:12345-192.168.204.60@tcp:2:lfs.0 00000001:00080000:0.0:1769074803.546654:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874606 is committed 00000001:00080000:0.0:1769074803.546667:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874605 is committed 00000001:00100000:0.0:1769074803.546684:0:26036:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00002000:00010000:1.0:1769074803.550069:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1769074803.550078:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb mtime from disk: 0 -> 1769074798 00002000:00010000:1.0:1769074803.550081:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb atime from disk: 0 -> 1769074798 00002000:00010000:1.0:1769074803.550083:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb ctime from disk: 0 -> 1769074803 00002000:00010000:1.0:1769074803.550086:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074803.550305:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2409:x1855007616966912:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 6918us (7099us total) trans 17179874606 rc 0/0 00000100:00100000:1.0:1769074803.550315:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14438 00000100:00100000:3.0:1769074803.553359:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.555959:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616967168 00000100:00100000:1.0:1769074803.555975:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616967168 00000100:00100000:1.0:1769074803.556147:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13406 00000100:00100000:1.0:1769074803.556159:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:63851:x1855007616967168:12345-192.168.204.60@tcp:55:lfs.0 00010000:00010000:1.0:1769074803.556225:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.556244:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.556253:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 4/0,1 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.556289:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 3/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.556302:0:8855:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 11 pid: 8855 timeout: 1891 lvb_type: 0 00000100:00100000:1.0:1769074803.556315:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625202560:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074803.556411:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.561732:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.562178:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616967296 00000100:00100000:3.0:1769074803.562189:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616967296 00010000:00010000:3.0:1769074803.562227:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 5/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 12 pid: 8855 timeout: 1891 lvb_type: 0 00000100:00100000:3.0:1769074803.562274:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7181 00000100:00100000:3.0:1769074803.562286:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e664050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:2409:x1855007616967296:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074803.562302:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074803.562308:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 5/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 12 pid: 8855 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.562322:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 12 pid: 8855 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.562345:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:3.0:1769074803.562349:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074803.562359:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074803.562366:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 4/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074803.562383:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074803.562450:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e664050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:2409:x1855007616967296:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 153us (713us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.562458:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7181 00010000:00010000:1.0:1769074803.563772:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866c00/0x7fe2323adfd5495e lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c449 expref: 11 pid: 8855 timeout: 1891 lvb_type: 0 00000100:00100000:1.0:1769074803.563847:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8855:1855007625202560:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074803.563872:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563879:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563884:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563930:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563937:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 2/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563943:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563949:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074803.563953:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 3/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563959:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 4/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563972:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:1.0:1769074803.563974:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1769074803.563979:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 2/0,0 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074803.563984:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867000/0x7fe2323adfd5498f lrc: 0/0,0 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000100:00100000:1.0:1769074803.564054:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616967168:12345-192.168.204.60@tcp:55:lfs.0 Request processed in 7897us (10704us total) trans 0 rc -22/-22 00000100:00100000:1.0:1769074803.564066:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13406 00000100:00100000:0.0:1769074803.568076:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.568734:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616967552 00000100:00100000:2.0:1769074803.568880:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616967552 00000100:00100000:2.0:1769074803.568953:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13407 00000100:00100000:2.0:1769074803.568964:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:63851:x1855007616967552:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074803.569022:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074803.569051:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd548d9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c411 expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.569081:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074803.569086:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074803.569275:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.569291:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.569302:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00010000:2.0:1769074803.569430:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000003748c3e2 reply state 00000000e240d971 transno 8589944033 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.569455:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074803.569462:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074803.569464:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074803.569467:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074803.569562:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:63851:x1855007616967552:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 599us (1505us total) trans 8589944033 rc 0/0 00000100:00100000:2.0:1769074803.569569:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13407 00000100:00100000:2.0:1769074803.579191:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.579280:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616967680 00000100:00100000:3.0:1769074803.579291:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616967680 00000100:00100000:3.0:1769074803.579349:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13408 00000100:00100000:3.0:1769074803.579359:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+11:63851:x1855007616967680:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074803.579382:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074803.579403:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c465 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.579461:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.579476:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074803.579495:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074803.579499:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074803.579503:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb600, rc 0) 00000100:00100000:3.0:1769074803.579553:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:63851:x1855007616967680:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 196us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.579561:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13408 00000100:00100000:0.0:1769074803.583962:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.584068:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616967808 00000100:00100000:1.0:1769074803.584078:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616967808 00000100:00100000:1.0:1769074803.584134:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14439 00000100:00100000:1.0:1769074803.584153:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:63851:x1855007616967808:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.584173:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.584193:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c46c expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.584224:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.584232:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.584234:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074803.584238:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.584275:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074803.584289:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 9 pid: 28955 timeout: 1891 lvb_type: 1 00000100:00100000:1.0:1769074803.584302:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625202688:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074803.584395:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.590524:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.590628:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616967936 00000100:00100000:2.0:1769074803.590638:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616967936 00010000:00010000:2.0:1769074803.590674:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 10 pid: 28955 timeout: 1891 lvb_type: 1 00000100:00100000:2.0:1769074803.590713:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7182 00000100:00100000:2.0:1769074803.590722:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+10:2408:x1855007616967936:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074803.590746:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074803.590765:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 10 pid: 28955 timeout: 1891 lvb_type: 1 00010000:00010000:2.0:1769074803.590777:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 10 pid: 28955 timeout: 1891 lvb_type: 1 00010000:00010000:2.0:1769074803.590842:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074803.590845:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074803.590850:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.590860:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.590892:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074803.590904:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625202816:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074803.590969:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc18e40 going to sleep for 11 seconds 00010000:00010000:1.0:1769074803.591536:0:8398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547866e00/0x7fe2323adfd54911 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c426 expref: 10 pid: 28955 timeout: 1891 lvb_type: 1 00000100:00100000:1.0:1769074803.591551:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625202688:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074803.591577:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.591586:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867000, rc 0) 00000100:00100000:1.0:1769074803.591632:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:63851:x1855007616967808:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 7481us (7680us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.591639:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14439 00000100:00100000:2.0:1769074803.607212:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475ded80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625202816:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074803.607240:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074803.607289:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616967936:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 16566us (16777us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074803.607300:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7182 00000100:00100000:2.0:1769074803.612039:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.612128:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968064 00000100:00100000:1.0:1769074803.612150:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968064 00000100:00100000:1.0:1769074803.612211:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14440 00000100:00100000:1.0:1769074803.612222:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2408:x1855007616968064:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:1.0:1769074803.612391:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074803.612396:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb mtime from disk: 1769074798 -> 1769074798 00002000:00010000:1.0:1769074803.612398:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb atime from disk: 0 -> 1769074798 00002000:00010000:1.0:1769074803.612401:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb ctime from disk: 1769074798 -> 1769074803 00002000:00010000:1.0:1769074803.612403:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074803.612473:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:2408:x1855007616968064:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 252us (444us total) trans 21474841276 rc 0/0 00000100:00100000:1.0:1769074803.612480:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14440 00000100:00100000:0.0:1769074803.618705:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.618915:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968320 00000100:00100000:2.0:1769074803.618926:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968320 00000100:00100000:2.0:1769074803.618987:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4706 00000100:00100000:2.0:1769074803.618997:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:63851:x1855007616968320:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:2.0:1769074803.619071:0:6515:0:(mdt_open.c:2313:mdt_close_handle_layouts()) ### [0x200000405:0x1fe:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd5493b lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c442 expref: 12 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.619115:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.619127:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.619150:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.619183:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.619197:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 12 pid: 6513 timeout: 1891 lvb_type: 0 00000100:00100000:2.0:1769074803.619213:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625202944:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074803.619300:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc18e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.623326:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074803.623417:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968448 00000100:00100000:3.0:1769074803.623427:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968448 00010000:00010000:3.0:1769074803.623477:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 13 pid: 6513 timeout: 1891 lvb_type: 0 00000100:00100000:3.0:1769074803.623516:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7183 00000100:00100000:3.0:1769074803.623526:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+13:2408:x1855007616968448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074803.623541:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074803.623547:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 13 pid: 6513 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.623559:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 13 pid: 6513 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.623581:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074803.623587:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074803.623597:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074803.623609:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074803.623626:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074803.623683:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+13:2408:x1855007616968448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 158us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.623690:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7183 00010000:00010000:2.0:1769074803.624167:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5499d lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c465 expref: 12 pid: 6513 timeout: 1891 lvb_type: 0 00000100:00100000:2.0:1769074803.624184:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625202944:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074803.624204:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.624212:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.624217:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.624240:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.624249:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.624256:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 4/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.624281:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 3/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074803.624293:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 4/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 11 pid: 10283 timeout: 1891 lvb_type: 0 00000100:00100000:2.0:1769074803.624307:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625203072:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074803.624376:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc18e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.630343:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.630464:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625203072:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074803.630492:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:3.0:1769074803.630879:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968576 00000100:00100000:3.0:1769074803.630889:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968576 00010000:00010000:3.0:1769074803.630926:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 4/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 12 pid: 10283 timeout: 1891 lvb_type: 0 00000100:00100000:3.0:1769074803.630967:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7184 00000100:00100000:3.0:1769074803.630977:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:2409:x1855007616968576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074803.630993:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074803.631001:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 4/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 12 pid: 10283 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.631012:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 3/0,0 mode: CR/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 12 pid: 10283 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.631034:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5497a lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c450 expref: 12 pid: 10283 timeout: 1891 lvb_type: 0 00010000:00010000:3.0:1769074803.631049:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:3.0:1769074803.631054:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074803.631066:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074803.631072:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 4/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074803.631101:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074803.631224:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074803.631232:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:0.0:1769074803.631526:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203200:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074803.631575:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074803.631603:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499c5e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.631622:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203200 00000100:00100000:0.0:1769074803.631628:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203200 00000100:00100000:0.0:1769074803.631657:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2668 00000100:00100000:0.0:1769074803.631664:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203200:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074803.631734:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203200:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 70us (158us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074803.631741:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2668 00000100:00100000:0.0:1769074803.631778:0:6515:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841273 gen 2 00000100:00100000:3.0:1769074803.631971:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:2409:x1855007616968576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 993us (1634us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.631978:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7184 00000004:00080000:0.0:1769074803.632539:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d103100, transno 21474841266 00000004:00080000:0.0:1769074803.632558:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a6680, transno 21474841267 00000004:00080000:0.0:1769074803.632562:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a7100, transno 21474841268 00000004:00080000:0.0:1769074803.632564:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ed880, transno 21474841269 00000004:00080000:0.0:1769074803.632567:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a4700, transno 21474841270 00000004:00080000:0.0:1769074803.632569:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a4000, transno 21474841271 00000004:00080000:0.0:1769074803.632571:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ef800, transno 21474841272 00000004:00080000:0.0:1769074803.632574:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a4a80, transno 21474841273 00000100:00100000:0.0:1769074803.632581:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203200:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074803.632648:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203328:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074803.632685:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074803.632711:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6499c56c0 going to sleep for 11 seconds 00000004:00080000:2.0:1769074803.633271:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 8 done 00000100:00100000:3.0:1769074803.635011:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203328 00000100:00100000:3.0:1769074803.635026:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203328 00000100:00100000:3.0:1769074803.635101:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2669 00000100:00100000:3.0:1769074803.635111:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203328:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1769074803.635243:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203328:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 132us (2555us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.635251:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2669 00000100:00100000:2.0:1769074803.635657:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203328:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074803.635750:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203456:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074803.635801:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074803.635857:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc18f00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074803.636299:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203456 00000100:00100000:3.0:1769074803.636306:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203456 00000100:00100000:3.0:1769074803.636352:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2670 00000100:00100000:3.0:1769074803.636360:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203456:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1769074803.636454:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203456:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 80us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074803.636461:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2670 00000100:00100000:2.0:1769074803.637047:0:6515:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874605 gen 2 00000004:00080000:2.0:1769074803.637055:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5461e5c00, transno 17179874597 00000004:00080000:2.0:1769074803.637070:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5461e7800, transno 17179874598 00000004:00080000:2.0:1769074803.637073:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759d500, transno 17179874599 00000004:00080000:2.0:1769074803.637076:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759d880, transno 17179874600 00000004:00080000:2.0:1769074803.637078:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b545109f80, transno 17179874601 00000004:00080000:2.0:1769074803.637080:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759c000, transno 17179874602 00000004:00080000:2.0:1769074803.637083:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759fb80, transno 17179874603 00000004:00080000:2.0:1769074803.637085:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547598000, transno 17179874604 00000004:00080000:2.0:1769074803.637088:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d0e4a80, transno 17179874605 00000100:00100000:2.0:1769074803.637094:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203456:0@lo:1000:mdt_rdpg00_000.0 00000040:00080000:2.0:1769074803.637302:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].82, 368 off86024 00000001:00080000:2.0:1769074803.637314:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944034 00000100:00100000:2.0:1769074803.637378:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203584:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074803.637419:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074803.637889:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc180c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074803.637910:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203584 00000100:00100000:2.0:1769074803.637916:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203584 00000100:00100000:2.0:1769074803.637957:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2671 00000100:00100000:2.0:1769074803.637964:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203584:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074803.638094:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6515:x1855007625203584:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 130us (674us total) trans 17179874607 rc 0/0 00000100:00100000:2.0:1769074803.638101:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2671 00000004:00080000:0.0:1769074803.638333:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 9 done 00000100:00100000:2.0:1769074803.638503:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dea00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203584:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074803.638536:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203712:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074803.638566:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074803.638590:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc180c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074803.639497:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203712 00000100:00100000:0.0:1769074803.639506:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203712 00000100:00100000:0.0:1769074803.639562:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2672 00000100:00100000:0.0:1769074803.639571:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625203712:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074803.639719:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6515:x1855007625203712:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 149us (1152us total) trans 21474841277 rc 0/0 00000100:00100000:0.0:1769074803.639728:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2672 00000100:00100000:2.0:1769074803.640144:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625203712:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1769074803.640216:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944035 00010000:00010000:2.0:1769074803.640250:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,1 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640259:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 2/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640266:0:6515:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640272:0:6515:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074803.640277:0:6515:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 3/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640284:0:6515:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 4/0,0 mode: EX/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640299:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:2.0:1769074803.640301:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1769074803.640306:0:6515:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 2/0,0 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640311:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd549b2 lrc: 0/0,0 mode: --/EX res: [0x200000bd1:0x1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640325:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640330:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640335:0:6515:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640343:0:6515:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074803.640345:0:6515:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640350:0:6515:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640356:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074803.640358:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1769074803.640360:0:6515:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640364:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd549ab lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074803.640374:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074803.640376:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074803.640383:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866400/0x7fe2323adfd5493b lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c442 expref: 13 pid: 8855 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074803.641043:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:63851:x1855007616968320:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 22043us (22344us total) trans 8589944036 rc 0/0 00000100:00100000:2.0:1769074803.641053:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4706 00000020:00080000:2.0:1769074803.641058:0:6515:0:(tgt_main.c:782:tgt_ses_key_exit()) total 3 transactions per RPC 00000100:00100000:2.0:1769074803.657025:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.657072:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.657095:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.657115:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.657141:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.657165:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074803.657489:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968832 00000100:00100000:1.0:1769074803.657500:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968832 00000100:00100000:1.0:1769074803.657561:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 355 00000100:00100000:1.0:1769074803.657571:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681596050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2409:x1855007616968832:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074803.657658:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681596050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2409:x1855007616968832:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 87us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.657666:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 355 00000100:00100000:1.0:1769074803.657702:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616969344 00000100:00100000:1.0:1769074803.657707:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616969344 00000100:00100000:1.0:1769074803.657740:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616969216 00000100:00100000:1.0:1769074803.657744:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616969216 00000100:00100000:1.0:1769074803.657769:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1484 00000100:00100000:1.0:1769074803.657776:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616969344:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074803.657844:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616969344:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 67us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.657851:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1484 00000100:00100000:1.0:1769074803.657865:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1485 00000100:00100000:1.0:1769074803.657872:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616969216:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074803.657897:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616969216:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (732us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.657903:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1485 00000100:00080000:1.0:1769074803.657924:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074803.657927:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074803.657940:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968704 00000100:00100000:1.0:1769074803.657944:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968704 00000100:00100000:1.0:1769074803.657973:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616969088 00000100:00100000:1.0:1769074803.657977:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616969088 00000100:00100000:1.0:1769074803.658000:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616968960 00000100:00100000:1.0:1769074803.658003:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616968960 00000100:00100000:1.0:1769074803.658018:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2371 00000100:00100000:1.0:1769074803.658024:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007616969088:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074803.658058:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007616969088:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 35us (963us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.658065:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2371 00000100:00100000:1.0:1769074803.658077:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2372 00000100:00100000:1.0:1769074803.658083:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616968960:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074803.658107:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616968960:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (972us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.658112:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2372 00000100:00100000:1.0:1769074803.658125:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13409 00000100:00100000:1.0:1769074803.658131:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e723050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+12:63851:x1855007616968704:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074803.658155:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074803.658174:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c473 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.658229:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.658243:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074803.658262:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074803.658265:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074803.658269:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867200, rc 0) 00000100:00100000:1.0:1769074803.658290:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e723050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+13:63851:x1855007616968704:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 159us (1276us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074803.658296:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13409 00000100:00080000:1.0:1769074803.658314:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074803.658317:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074803.664206:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074803.664283:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616969600 00000100:00100000:2.0:1769074803.664295:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616969600 00000100:00100000:2.0:1769074803.664358:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4707 00000100:00100000:2.0:1769074803.664369:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+13:63851:x1855007616969600:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074803.664628:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+14:63851:x1855007616969600:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 260us (432us total) trans 8589944037 rc 0/0 00000100:00100000:2.0:1769074803.664636:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4707 00000100:00100000:1.0:1769074803.677358:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074803.677474:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616969856 00000100:00100000:0.0:1769074803.677485:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616969856 00000100:00100000:0.0:1769074803.677548:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4708 00000100:00100000:0.0:1769074803.677559:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+14:63851:x1855007616969856:12345-192.168.204.60@tcp:35:lfs.0 00000004:00080000:0.0:1769074803.677775:0:15947:0:(mdd_object.c:3993:mdd_close()) Object [0x200000bd1:0x1:0x0] is deleted from orphan list, OSS objects to be destroyed. 00010000:00010000:0.0:1769074803.677918:0:15947:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 3/0,1 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074803.677936:0:15947:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074803.677947:0:15947:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 3/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074803.677956:0:15947:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 4/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074803.677962:0:15947:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 4/0,1 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074803.677968:0:15947:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 3/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074803.677997:0:15947:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074803.678327:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074803.678335:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 2/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074803.678346:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074803.678350:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 2/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074803.678357:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 3/0,0 mode: PW/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074803.678370:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:3.0:1769074803.678373:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1769074803.678378:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 1/0,0 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074803.678384:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd549c0 lrc: 0/0,0 mode: --/PW res: [0x200000bd1:0x1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074803.678395:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074803.678397:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074803.678398:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074803.685388:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+15:63851:x1855007616969856:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 7818us (8027us total) trans 8589944038 rc 0/0 00000100:00100000:1.0:1769074803.685405:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4708 00010000:00010000:0.0:1769074804.125525:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1769074804.127340:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968348 is committed 00000001:00100000:1.0:1769074804.127357:0:7431:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0001: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000100:00100000:0.0:1769074804.637003:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074804.637030:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625203840:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074804.637126:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074804.637258:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203840 00000100:00100000:3.0:1769074804.637267:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203840 00000100:00100000:3.0:1769074804.637324:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2673 00000100:00100000:3.0:1769074804.637334:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625203840:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074804.637368:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074804.637428:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625203840:12345-0@lo:41:osp-pre-1-0.0 Request processed in 95us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074804.637435:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2673 00000100:00100000:2.0:1769074804.637557:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074804.637567:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625203968:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074804.637599:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074804.637672:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625203968 00000100:00100000:3.0:1769074804.637677:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625203968 00000100:00100000:3.0:1769074804.637712:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2674 00000100:00100000:3.0:1769074804.637718:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625203968:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074804.637735:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00100000:3.0:1769074804.637768:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625203968:12345-0@lo:41:osp-pre-0-1.0 Request processed in 50us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074804.637802:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2674 00000100:00100000:2.0:1769074804.637907:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625203968:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074804.637917:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625203840:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074805.149991:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074805.150013:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625204096:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074805.150080:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1769074805.150125:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074805.150139:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204096 00000100:00100000:2.0:1769074805.150147:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204096 00000100:00100000:2.0:1769074805.150206:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1192 00000100:00100000:2.0:1769074805.150216:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625204096:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074805.150239:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 499651904 pending 0 free 3913945088 avail 3678179328 00000100:00100000:2.0:1769074805.150283:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625204096:12345-0@lo:13:osp-pre-0-1.0 Request processed in 69us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074805.150290:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1192 00000100:00080000:2.0:1769074805.150314:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074805.150353:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625204096:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074805.152159:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625204224:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074805.152226:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074805.155331:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625204352:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074805.155407:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074805.155554:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204224 00000100:00100000:0.0:1769074805.155563:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204224 00000100:00100000:0.0:1769074805.155623:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204352 00000100:00100000:0.0:1769074805.155627:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204352 00000100:00100000:0.0:1769074805.155652:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1193 00000100:00100000:0.0:1769074805.155661:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625204224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074805.155690:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:0.0:1769074805.155734:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625204224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (3507us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074805.155741:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1193 00000100:00100000:0.0:1769074805.155758:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1194 00000100:00100000:0.0:1769074805.155775:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625204352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074805.155784:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 499651904 pending 0 free 3913945088 avail 3678179328 00000100:00100000:0.0:1769074805.155835:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625204352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074805.155841:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1194 00000100:00100000:1.0:1769074805.156056:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625204352:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074805.156103:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625204224:0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1769074805.158190:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074805.158199:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:02000400:0.0:1769074805.611229:0:36875:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: mirror extend done 00000100:00100000:0.0:1769074805.661110:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625204480:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074805.661182:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074805.661366:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204480 00000100:00100000:1.0:1769074805.661376:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204480 00000100:00100000:1.0:1769074805.661459:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1195 00000100:00100000:1.0:1769074805.661469:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625204480:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1769074805.661492:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:1.0:1769074805.661539:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625204480:12345-0@lo:13:osp-pre-1-1.0 Request processed in 71us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074805.661545:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1195 00000100:00080000:0.0:1769074805.664709:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074805.664783:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625204480:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1769074806.182295:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00020000:3.0:1769074806.702707:0:6512:0:(mdt_open.c:1759:mdt_reint_open()) cfs_fail_timeout interrupted 00010000:00010000:3.0:1769074806.719372:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd549c7 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719398:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd549c7 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719412:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd549c7 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074806.719427:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00010000:00010000:3.0:1769074806.719551:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719569:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719574:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719581:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719586:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074806.719591:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719602:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719617:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074806.719620:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074806.719624:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719630:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd548a8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719640:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719646:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719651:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719656:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074806.719658:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719664:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719670:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b64effa500) 00010000:00010000:3.0:1769074806.719672:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074806.719675:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719680:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd548af lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074806.719746:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd548a1 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c403 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074806.719761:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd549c7 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c403 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074806.719768:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074806.719770:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074806.719772:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbe00, rc 0) 00000100:00100000:3.0:1769074806.720040:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:63717:x1855007616962304:12345-192.168.204.60@tcp:101:dd.0 Request processed in 6332806us (6336279us total) trans 8589944039 rc 0/0 00000100:00100000:3.0:1769074806.720051:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13395 00000100:00100000:1.0:1769074806.774290:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074806.774368:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616970112 00000100:00100000:2.0:1769074806.774378:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616970112 00000100:00100000:2.0:1769074806.774450:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13410 00000100:00100000:2.0:1769074806.774461:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:63717:x1855007616970112:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074806.774484:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074806.774487:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074806.774511:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd549c7 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c403 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.774527:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074806.774531:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074806.774545:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd549d5 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c47a expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.774590:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074806.774603:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.774611:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 4/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074806.774643:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.774655:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 14 pid: 6514 timeout: 1894 lvb_type: 0 00000100:00100000:2.0:1769074806.774668:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625204608:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074806.774762:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00080000:1.0:1769074806.774838:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074806.782625:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074806.782673:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616970240 00000100:00100000:1.0:1769074806.782683:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616970240 00010000:00010000:1.0:1769074806.782733:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 15 pid: 6514 timeout: 1894 lvb_type: 0 00000100:00100000:1.0:1769074806.782783:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7185 00000100:00100000:1.0:1769074806.782792:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+15:2409:x1855007616970240:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074806.782829:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074806.782835:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 15 pid: 6514 timeout: 1894 lvb_type: 0 00010000:00010000:1.0:1769074806.782848:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 15 pid: 6514 timeout: 1894 lvb_type: 0 00010000:00010000:1.0:1769074806.782869:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074806.782875:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074806.782884:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074806.782891:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074806.782910:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074806.782959:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+15:2409:x1855007616970240:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 168us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074806.782968:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7185 00000100:00080000:1.0:1769074806.782996:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1769074806.785537:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867200/0x7fe2323adfd549b9 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c473 expref: 14 pid: 6514 timeout: 1894 lvb_type: 0 00000100:00100000:0.0:1769074806.785559:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625204608:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074806.785584:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074806.790156:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074806.790173:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000040:00080000:1.0:1769074806.790403:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2459, 88 off210152 00000004:00080000:1.0:1769074806.790414:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1794632606382ns, next at 1788871191006ns, rc = 0 00000040:00080000:1.0:1769074806.790444:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2460, 88 off210240 00000004:00080000:1.0:1769074806.790449:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1794632642272ns, next at 1788871191006ns, rc = 0 00000040:00080000:1.0:1769074806.790469:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2465, 88 off210920 00000004:00080000:1.0:1769074806.790475:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1794632667836ns, next at 1788871251280ns, rc = 0 00000001:00080000:1.0:1769074806.792391:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944040 is committed 00000004:00080000:1.0:1769074806.792402:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1961 committed 00000100:00100000:2.0:1769074806.799381:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625204736:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074806.799453:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074806.799516:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204736 00000100:00100000:2.0:1769074806.799523:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204736 00000100:00100000:2.0:1769074806.799571:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14441 00000100:00100000:2.0:1769074806.799580:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625204736:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1769074806.799720:0:9048:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1769074806.799724:0:9048:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb mtime from disk: 1769074798 -> 1769074798 00002000:00010000:2.0:1769074806.799726:0:9048:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb atime from disk: 1769074798 -> 1769074798 00002000:00010000:2.0:1769074806.799728:0:9048:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb ctime from disk: 1769074803 -> 1769074803 00002000:00010000:2.0:1769074806.799730:0:9048:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c5:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074806.799780:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625204736:12345-0@lo:2:osp-syn-1-0.0 Request processed in 201us (326us total) trans 17179874608 rc 0/0 00000100:00100000:2.0:1769074806.799788:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14441 00000004:00080000:2.0:1769074806.799853:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546f91180/2, rc 0, transno 17179874608 00000100:00100000:2.0:1769074806.799858:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625204736:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074806.801831:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1958 committed 00000004:00080000:1.0:1769074806.801848:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1957 committed 00000001:00080000:1.0:1769074806.801870:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944038 is committed 00000020:00080000:1.0:1769074806.801884:0:32377:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff89b6499c5e40 refcount 2 committed 0 result 0 batchid 4294969364 00000020:00080000:1.0:1769074806.801889:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b6499f2e00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b67fc18e40 00000020:00080000:1.0:1769074806.801892:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b644c73500 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b6499c5b40 00000020:00080000:1.0:1769074806.801894:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b644c73700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff89b54367c600 00000020:00080000:1.0:1769074806.801897:0:32377:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].82 00010000:00010000:1.0:1769074806.801999:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802008:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802015:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802022:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074806.802027:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802035:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802051:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1:0x0].0x0 (ffff89b6439ad500) 00010000:00010000:1.0:1769074806.802054:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074806.802059:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 2/0,0 mode: --/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802065:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54973 lrc: 0/0,0 mode: --/TXN res: [0x200000bd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:1.0:1769074806.802105:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074806.802112:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074806.802117:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074806.802122:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074806.802125:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074806.802130:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074806.802138:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074806.802140:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074806.802144:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074806.802149:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54996 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000020:00080000:1.0:1769074806.802167:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294969364 committed batchid 4294969363 00000020:00080000:1.0:1769074806.802230:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969363 00000100:00100000:1.0:1769074806.802250:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625204864:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074806.802310:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074806.802344:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074806.802349:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625204992:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074806.802369:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1769074806.802425:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c47a expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074806.802434:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074806.802436:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:1.0:1769074806.802450:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd549d5 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c47a expref: 8 pid: 6512 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1769074806.802465:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c47a expref: 7 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074806.802492:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074806.802494:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:1.0:1769074806.802497:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88400, rc 0) 00000100:00100000:1.0:1769074806.802547:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:63717:x1855007616970112:12345-192.168.204.60@tcp:101:dd.0 Request processed in 28088us (28268us total) trans 8589944040 rc 0/0 00000100:00100000:1.0:1769074806.802555:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13410 00000100:00100000:2.0:1769074806.803157:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204864 00000100:00100000:2.0:1769074806.803167:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204864 00000100:00100000:2.0:1769074806.803207:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625204992 00000100:00100000:2.0:1769074806.803210:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625204992 00000100:00100000:2.0:1769074806.803239:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14442 00000100:00100000:2.0:1769074806.803248:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625204864:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074806.803370:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074806.803374:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb mtime from disk: 1769074798 -> 1769074798 00002000:00010000:2.0:1769074806.803376:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb atime from disk: 1769074798 -> 1769074798 00002000:00010000:2.0:1769074806.803379:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb ctime from disk: 1769074803 -> 1769074803 00002000:00010000:2.0:1769074806.803381:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:2.0:1769074806.803429:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625204864:12345-0@lo:2:osp-syn-0-0.0 Request processed in 181us (1118us total) trans 21474841278 rc 0/0 00000100:00100000:2.0:1769074806.803436:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14442 00000100:00100000:2.0:1769074806.803451:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14443 00000100:00100000:2.0:1769074806.803457:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625204992:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074806.803514:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074806.803517:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb mtime from disk: 1769074798 -> 1769074798 00002000:00010000:2.0:1769074806.803519:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb atime from disk: 1769074798 -> 1769074798 00002000:00010000:2.0:1769074806.803522:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb ctime from disk: 1769074803 -> 1769074803 00002000:00010000:2.0:1769074806.803524:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c2:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:2.0:1769074806.803549:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625204992:12345-0@lo:2:osp-syn-0-0.0 Request processed in 92us (1179us total) trans 21474841279 rc 0/0 00000100:00100000:2.0:1769074806.803555:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14443 00000004:00080000:2.0:1769074806.803604:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54441d880/2, rc 0, transno 21474841279 00000100:00100000:2.0:1769074806.803610:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625204992:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074806.803620:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54441f480/2, rc 0, transno 21474841278 00000100:00100000:2.0:1769074806.803623:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625204864:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074806.814374:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074806.814926:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616970368 00000100:00100000:0.0:1769074806.814938:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616970368 00000100:00100000:0.0:1769074806.814998:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13411 00000100:00100000:0.0:1769074806.815009:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:63717:x1855007616970368:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:0.0:1769074806.815032:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074806.815051:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd549e3 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c481 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.815214:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd549ea lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074806.815229:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd549ea lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074806.815240:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd549ea lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074806.815346:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd549e3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c481 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.815364:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd549ea lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c481 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.815372:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:0.0:1769074806.815375:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1769074806.815378:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165a00, rc 0) 00000100:00100000:0.0:1769074806.815882:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:63717:x1855007616970368:12345-192.168.204.60@tcp:101:dd.0 Request processed in 872us (1516us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074806.815891:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13411 00000100:00100000:1.0:1769074806.823415:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074806.826414:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616970496 00000100:00100000:0.0:1769074806.826429:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616970496 00000100:00100000:1.0:1769074806.827092:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8739 00000100:00100000:1.0:1769074806.827104:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616970496:12345-192.168.204.60@tcp:4:dd.0 00010000:00010000:1.0:1769074806.827146:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074806.827166:0:8404:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 8 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074806.827176:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074806.827179:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074806.827183:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074806.827220:0:8404:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 8 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074806.827235:0:8404:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 8 pid: 8398 timeout: 1894 lvb_type: 0 00000100:00100000:1.0:1769074806.827249:0:8404:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8404:1855007625205120:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074806.827350:0:8404:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6760933c0 going to sleep for 11 seconds 00000100:00080000:1.0:1769074806.827370:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074806.841239:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074806.841306:0:8404:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8404:1855007625205120:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074806.841338:0:8404:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074806.841371:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616970624 00000100:00100000:0.0:1769074806.841378:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616970624 00010000:00010000:0.0:1769074806.841420:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 9 pid: 8398 timeout: 1894 lvb_type: 0 00000100:00100000:0.0:1769074806.841451:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7186 00000100:00100000:0.0:1769074806.841460:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+9:2409:x1855007616970624:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074806.841475:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074806.841499:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 9 pid: 8398 timeout: 1894 lvb_type: 0 00010000:00010000:0.0:1769074806.841511:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 9 pid: 8398 timeout: 1894 lvb_type: 0 00010000:00010000:0.0:1769074806.841536:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867000/0x7fe2323adfd549a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c46c expref: 9 pid: 8398 timeout: 1894 lvb_type: 0 00010000:00010000:0.0:1769074806.841563:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074806.841566:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074806.841571:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.841581:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.841603:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074806.841657:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+8:2409:x1855007616970624:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 198us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074806.841666:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7186 00000100:00080000:0.0:1769074806.841688:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1769074806.841699:0:8404:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.841706:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000001:00080000:3.0:1769074806.876137:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841280 is committed 00000001:00080000:3.0:1769074806.876152:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841279 is committed 00000001:00080000:3.0:1769074806.876161:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841276 is committed 00002000:00010000:2.0:1769074806.886986:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb mtime from disk: 1769074798 -> 1769074806 00002000:00010000:2.0:1769074806.886995:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c1:0x0] updating lvb ctime from disk: 1769074803 -> 1769074806 00010000:00010000:2.0:1769074806.887010:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.887021:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.887030:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.887038:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074806.887043:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.887051:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.887071:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.887078:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b547867c00/0x7fe2323adfd549f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074806.890942:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616970496:12345-192.168.204.60@tcp:4:dd.0 Request processed in 63829us (67524us total) trans 21474841280 rc 0/0 00000100:00100000:2.0:1769074806.890957:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8739 00000100:00100000:2.0:1769074806.952800:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074806.953009:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616970752 00000100:00100000:0.0:1769074806.953020:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616970752 00000100:00100000:0.0:1769074806.953101:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8740 00000100:00100000:0.0:1769074806.953111:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616970752:12345-192.168.204.60@tcp:4:dd.0 00002000:00010000:0.0:1769074806.953160:0:8404:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c1:0x0] initial LVB size: 2097152, mtime: 0x6971f076, atime: 0x6971f06e, ctime: 0x6971f076, blocks: 0x1000 00010000:00010000:0.0:1769074806.953170:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.953184:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074806.953186:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074806.953192:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074806.953206:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000001:00080000:3.0:1769074806.969332:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841281 is committed 00010000:00010000:2.0:1769074806.969550:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.969648:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.969657:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.969663:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074806.969667:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.969676:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.969695:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074806.969701:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b682165000/0x7fe2323adfd549f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074806.969952:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616970752:12345-192.168.204.60@tcp:4:dd.0 Request processed in 16837us (17156us total) trans 21474841281 rc 0/0 00000100:00100000:2.0:1769074806.969963:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8740 00000100:00100000:3.0:1769074806.974594:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074806.974712:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616971008 00000100:00100000:1.0:1769074806.974724:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616971008 00000100:00100000:1.0:1769074806.974793:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4709 00000100:00100000:1.0:1769074806.974802:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:63717:x1855007616971008:12345-192.168.204.60@tcp:35:dd.0 00000100:00100000:1.0:1769074806.975143:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:63717:x1855007616971008:12345-192.168.204.60@tcp:35:dd.0 Request processed in 340us (558us total) trans 8589944041 rc 0/0 00000100:00100000:1.0:1769074806.975151:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4709 00010000:00010000:0.0:1769074807.198932:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074807.710614:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625205376:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.710616:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625205248:0@lo:400:kworker.0 00000100:00100000:1.0:1769074807.710708:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625205504:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.710821:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074807.710854:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625205888:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.710892:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074807.710909:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625206400:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.710929:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074807.710963:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625205632:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.710989:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074807.711007:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625206144:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711023:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074807.711038:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625206656:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711057:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074807.711089:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625205376 00000100:00100000:0.0:1769074807.711097:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625205376 00000100:00100000:0.0:1769074807.711139:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206400 00000100:00100000:0.0:1769074807.711142:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206400 00000100:00100000:0.0:1769074807.711162:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2675 00000100:00100000:0.0:1769074807.711169:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625205376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711221:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625205376:12345-0@lo:400:kworker.0 Request processed in 53us (417us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.711229:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2675 00000100:00100000:0.0:1769074807.711246:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2676 00000100:00100000:0.0:1769074807.711252:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625206400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711279:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625206400:12345-0@lo:400:kworker.0 Request processed in 27us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.711284:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2676 00000100:00100000:0.0:1769074807.711299:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625205888 00000100:00100000:0.0:1769074807.711303:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625205888 00000100:00100000:0.0:1769074807.711333:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206144 00000100:00100000:0.0:1769074807.711336:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206144 00000100:00100000:0.0:1769074807.711356:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2373 00000100:00100000:0.0:1769074807.711361:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625205888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711390:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625205888:12345-0@lo:400:kworker.0 Request processed in 29us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.711395:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2373 00000100:00100000:0.0:1769074807.711407:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2374 00000100:00100000:0.0:1769074807.711412:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625206144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711443:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625206144:12345-0@lo:400:kworker.0 Request processed in 31us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.711449:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2374 00000100:00100000:0.0:1769074807.711470:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625205632 00000100:00100000:0.0:1769074807.711473:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625205632 00000100:00100000:0.0:1769074807.711497:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206656 00000100:00100000:0.0:1769074807.711500:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206656 00000100:00100000:0.0:1769074807.711520:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1486 00000100:00100000:0.0:1769074807.711525:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625205632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711555:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625205632:12345-0@lo:400:kworker.0 Request processed in 29us (565us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.711560:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1486 00000100:00100000:0.0:1769074807.711570:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1487 00000100:00100000:0.0:1769074807.711576:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625206656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711601:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625206656:12345-0@lo:400:kworker.0 Request processed in 26us (543us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.711606:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1487 00000100:00080000:0.0:1769074807.711631:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074807.711634:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074807.711637:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074807.711677:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625205632:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711688:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625206144:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711696:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625206656:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711739:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625205376:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711748:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625205888:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.711756:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625206400:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.720458:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074807.720535:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625205760:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.720585:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074807.720611:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625206272:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.720635:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074807.720654:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625206784:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.720674:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074807.720738:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206272 00000100:00100000:3.0:1769074807.720746:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206272 00000100:00100000:3.0:1769074807.720797:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206784 00000100:00100000:3.0:1769074807.720801:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206784 00000100:00100000:3.0:1769074807.720851:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2375 00000100:00100000:3.0:1769074807.720860:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625206272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.720915:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625206272:12345-0@lo:400:kworker.0 Request processed in 55us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074807.720922:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2375 00000100:00100000:3.0:1769074807.720941:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2376 00000100:00100000:3.0:1769074807.720946:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625206784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.720989:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625206784:12345-0@lo:400:kworker.0 Request processed in 43us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074807.720995:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2376 00000100:00100000:3.0:1769074807.721013:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625205760 00000100:00100000:3.0:1769074807.721017:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625205760 00000100:00100000:3.0:1769074807.721045:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1488 00000100:00100000:3.0:1769074807.721051:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625205760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.721703:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625205760:12345-0@lo:400:kworker.0 Request processed in 649us (1114us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074807.721711:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1488 00000100:00080000:3.0:1769074807.721756:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074807.721761:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074807.721840:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625205760:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.721856:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625206272:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.721865:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625206784:0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.724281:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625205248 00000100:00100000:0.0:1769074807.724298:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625205248 00000100:00100000:0.0:1769074807.724365:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1769074807.724376:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681594c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625205248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074807.724471:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681594c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625205248:12345-0@lo:400:kworker.0 Request processed in 97us (4036us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074807.724479:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1769074807.724504:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625205248:0@lo:400:kworker.0 00000100:00100000:1.0:1769074807.726309:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074807.726358:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625206016:0@lo:400:kworker.0 00000100:00100000:1.0:1769074807.726394:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074807.726412:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625206528:0@lo:400:kworker.0 00000100:00100000:1.0:1769074807.726437:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074807.726785:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625205504 00000100:00100000:3.0:1769074807.726794:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625205504 00000100:00100000:3.0:1769074807.726859:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206016 00000100:00100000:3.0:1769074807.726863:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206016 00000100:00100000:3.0:1769074807.726893:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2377 00000100:00100000:3.0:1769074807.726901:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625205504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.726985:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625205504:12345-0@lo:400:kworker.0 Request processed in 84us (682us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074807.726993:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2377 00000100:00100000:3.0:1769074807.727009:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2378 00000100:00100000:3.0:1769074807.727017:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625206016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.727046:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625206016:12345-0@lo:400:kworker.0 Request processed in 30us (651us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074807.727051:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2378 00000100:00100000:3.0:1769074807.727086:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625206528 00000100:00100000:3.0:1769074807.727232:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625206528 00000100:00100000:3.0:1769074807.727267:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1489 00000100:00100000:3.0:1769074807.727274:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625206528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.727305:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625206528:12345-0@lo:400:kworker.0 Request processed in 32us (867us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074807.727310:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1489 00000100:00080000:3.0:1769074807.727328:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074807.727332:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074807.727373:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625205504:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.727382:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625206016:0@lo:400:kworker.0 00000100:00100000:3.0:1769074807.727390:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841279 gen 2 00000004:00080000:3.0:1769074807.727395:0:3651:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 21474841277 committed transno 21474841279 00000004:00080000:3.0:1769074807.727414:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54441f480, transno 21474841278 00000004:00080000:3.0:1769074807.727426:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54441d880, transno 21474841279 00000100:00100000:3.0:1769074807.727431:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625206528:0@lo:400:kworker.0 00010000:00010000:2.0:1769074808.224385:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1769074808.600153:0:36972:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: dd completed 00000100:00100000:2.0:1769074808.686979:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074808.687066:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616971136 00000100:00100000:3.0:1769074808.687076:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616971136 00000100:00100000:3.0:1769074808.687154:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13412 00000100:00100000:3.0:1769074808.687164:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64134:x1855007616971136:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074808.687187:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074808.687209:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd549ff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c488 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.687284:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54a06 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074808.687298:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54a06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074808.687308:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54a06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074808.687357:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1769074808.687362:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074808.687381:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54a06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074808.687406:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd549ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c488 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.687428:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54a06 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c488 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.687436:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074808.687439:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074808.687443:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba800, rc 0) 00000100:00100000:3.0:1769074808.687505:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64134:x1855007616971136:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 342us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074808.687512:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13412 00000100:00080000:1.0:1769074808.688509:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074808.692331:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074808.699422:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616971392 00000100:00100000:1.0:1769074808.699437:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616971392 00000100:00100000:1.0:1769074808.699514:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13413 00000100:00100000:1.0:1769074808.699523:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64134:x1855007616971392:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074808.699541:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074808.699560:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54a0d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c48f expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.699625:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699639:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699649:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699666:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699673:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699679:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699763:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699769:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699775:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699781:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699786:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074808.699790:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699798:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699824:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074808.699827:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074808.699832:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699838:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a14 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699853:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699859:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699864:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699869:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074808.699871:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699877:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699882:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b681697a00) 00010000:00010000:1.0:1769074808.699885:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074808.699887:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699892:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a1b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.699920:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54a0d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c48f expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.699928:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074808.699931:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074808.699934:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54a0d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c48f expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.699946:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867a00, rc 0) 00000100:00100000:1.0:1769074808.700012:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64134:x1855007616971392:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 489us (7691us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074808.700020:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13413 00000100:00100000:1.0:1769074808.714073:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074808.714425:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616971520 00000100:00100000:2.0:1769074808.714436:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616971520 00000100:00100000:2.0:1769074808.714496:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13414 00000100:00100000:2.0:1769074808.714506:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64134:x1855007616971520:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074808.714581:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714599:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714611:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714630:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714637:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714643:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714732:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714740:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714747:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074808.714786:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00000004:00200000:2.0:1769074808.714791:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074808.714832:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714847:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714853:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714858:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714864:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714873:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074808.714876:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714882:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714894:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074808.714897:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074808.714901:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714906:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54a22 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714920:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714925:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714931:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714935:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074808.714938:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714942:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714949:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b65436a900) 00010000:00010000:2.0:1769074808.714952:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074808.714954:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714959:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a29 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714974:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714980:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714985:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714989:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074808.714992:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.714998:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.715004:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074808.715006:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074808.715009:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074808.715013:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a30 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000100:00100000:2.0:1769074808.715100:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64134:x1855007616971520:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 594us (1047us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074808.715108:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13414 00000100:00100000:0.0:1769074808.763194:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074808.763308:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616971776 00000100:00100000:1.0:1769074808.763319:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616971776 00000100:00100000:1.0:1769074808.763375:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13415 00000100:00100000:1.0:1769074808.763386:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64137:x1855007616971776:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074808.763408:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074808.763428:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54a37 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c496 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.763505:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.763520:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.763531:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.763544:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.763551:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.763556:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767496:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767507:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767513:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767521:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767527:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547866600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074808.767531:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767539:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767559:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074808.767561:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074808.767566:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767571:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547866600/0x7fe2323adfd54a3e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767584:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767590:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767596:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767601:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b547867c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074808.767604:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767609:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767616:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b681697a00) 00010000:00010000:1.0:1769074808.767618:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074808.767621:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767625:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867c00/0x7fe2323adfd54a45 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074808.767680:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54a37 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c496 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.767689:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074808.767692:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074808.767695:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b547867a00/0x7fe2323adfd54a37 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c496 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.767714:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b547867a00, rc 0) 00000100:00100000:1.0:1769074808.767790:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64137:x1855007616971776:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4405us (4606us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074808.767801:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13415 00000100:00100000:3.0:1769074808.777061:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.777303:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616971904 00000100:00100000:0.0:1769074808.777313:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616971904 00000100:00100000:0.0:1769074808.777376:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 357 00000100:00100000:0.0:1769074808.777386:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681597850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2408:x1855007616971904:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.777483:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681597850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2408:x1855007616971904:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 98us (431us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.777492:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 357 00000100:00100000:0.0:1769074808.785521:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785565:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785588:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785623:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785642:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785660:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785678:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785699:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.785741:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972160 00000100:00100000:0.0:1769074808.785750:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972160 00000100:00100000:0.0:1769074808.785840:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972672 00000100:00100000:0.0:1769074808.785844:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972672 00000100:00100000:0.0:1769074808.785863:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972288 00000100:00100000:0.0:1769074808.785867:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972288 00000100:00100000:0.0:1769074808.785887:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972800 00000100:00100000:0.0:1769074808.785890:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972800 00000100:00100000:0.0:1769074808.785913:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1490 00000100:00100000:0.0:1769074808.785924:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616972160:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786002:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616972160:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 80us (436us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786011:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1490 00000100:00100000:0.0:1769074808.786029:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1491 00000100:00100000:0.0:1769074808.786035:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616972672:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786060:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616972672:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (436us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786067:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1491 00000100:00100000:0.0:1769074808.786078:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1492 00000100:00100000:0.0:1769074808.786085:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616972288:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786106:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616972288:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 22us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786113:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1492 00000100:00100000:0.0:1769074808.786125:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1493 00000100:00100000:0.0:1769074808.786131:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616972800:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786152:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616972800:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 22us (453us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786158:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1493 00000100:00080000:0.0:1769074808.786179:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074808.786184:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074808.786216:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972928 00000100:00100000:0.0:1769074808.786220:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972928 00000100:00100000:0.0:1769074808.786243:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972416 00000100:00100000:0.0:1769074808.786246:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972416 00000100:00100000:0.0:1769074808.786264:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972032 00000100:00100000:0.0:1769074808.786267:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972032 00000100:00100000:0.0:1769074808.786276:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616972544 00000100:00100000:0.0:1769074808.786278:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616972544 00000100:00100000:0.0:1769074808.786303:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2379 00000100:00100000:0.0:1769074808.786310:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616972416:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786341:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616972416:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 33us (752us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786350:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2379 00000100:00100000:0.0:1769074808.786362:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2380 00000100:00100000:0.0:1769074808.786367:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616972032:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786391:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616972032:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (748us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786397:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2380 00000100:00100000:0.0:1769074808.786408:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2381 00000100:00100000:0.0:1769074808.786413:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616972544:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074808.786435:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616972544:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 22us (757us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.786441:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2381 00000100:00100000:0.0:1769074808.786453:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13416 00000100:00100000:0.0:1769074808.786459:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64137:x1855007616972928:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074808.786531:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786549:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786561:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786575:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786581:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786587:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786672:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786680:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786687:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074808.786737:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00000004:00200000:0.0:1769074808.786742:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074808.786755:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786768:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786774:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786779:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786785:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786793:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074808.786797:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786833:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786846:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074808.786849:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074808.786854:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786860:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54a4c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786873:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786879:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786885:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786890:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074808.786892:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786898:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786905:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b643a80200) 00010000:00010000:0.0:1769074808.786907:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074808.786909:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786914:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54a53 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786927:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786932:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786937:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786941:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074808.786943:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786948:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786953:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:0.0:1769074808.786955:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074808.786958:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 2/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074808.786962:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54a5a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:0.0:1769074808.786997:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64137:x1855007616972928:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 538us (1487us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.787004:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13416 00000100:00080000:0.0:1769074808.787023:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074808.787026:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074808.835203:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074808.835323:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616973056 00000100:00100000:3.0:1769074808.835335:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616973056 00000100:00100000:3.0:1769074808.835390:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13417 00000100:00100000:3.0:1769074808.835399:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64143:x1855007616973056:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074808.835418:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074808.835440:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54a61 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c49d expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.835507:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835523:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835533:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835551:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835557:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835563:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835648:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835658:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835665:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:3.0:1769074808.835714:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00000004:00200000:3.0:1769074808.835719:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074808.835738:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835750:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835759:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835764:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835770:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835779:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835785:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074808.835790:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835798:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835836:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074808.835840:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074808.835845:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835851:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54a68 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835861:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835866:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835872:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835877:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074808.835880:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835886:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835895:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b6439ad500) 00010000:00010000:3.0:1769074808.835898:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074808.835900:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835905:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54a6f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.835932:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54a61 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c49d expref: 12 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.835945:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c49d expref: 11 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.835951:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074808.835954:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074808.835957:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb200, rc 0) 00000100:00100000:3.0:1769074808.836010:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:64143:x1855007616973056:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 611us (817us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074808.836018:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13417 00000100:00100000:3.0:1769074808.845063:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074808.845100:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616973184 00000100:00100000:3.0:1769074808.845111:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616973184 00000100:00100000:3.0:1769074808.845163:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14444 00000100:00100000:3.0:1769074808.845172:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616973184:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074808.845189:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074808.845209:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4a4 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074808.845238:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c1:0x0] initial LVB size: 2097152, mtime: 0x6971f076, atime: 0x6971f06e, ctime: 0x6971f076, blocks: 0x1000 00010000:00010000:3.0:1769074808.845258:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074808.845260:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074808.845268:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 6 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074808.845284:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 6 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074808.845312:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febba00, rc 0) 00000100:00100000:3.0:1769074808.845366:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616973184:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 194us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074808.845373:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14444 00000100:00100000:2.0:1769074808.864336:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074808.864498:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616973440 00000100:00100000:2.0:1769074808.864508:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616973440 00000100:00100000:2.0:1769074808.866242:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13418 00000100:00100000:2.0:1769074808.866262:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:64143:x1855007616973440:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074808.866288:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074808.866311:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a84 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4ab expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.866423:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a8b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074808.866439:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a8b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074808.866450:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a8b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074808.866462:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00010000:00010000:2.0:1769074808.866568:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54a84 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ab expref: 14 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.866588:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a8b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ab expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.866597:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:2.0:1769074808.866600:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074808.866604:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88000, rc 0) 00000100:00100000:2.0:1769074808.866675:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:64143:x1855007616973440:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 422us (2349us total) trans 8589944042 rc 0/0 00000100:00100000:2.0:1769074808.866683:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13418 00000100:00100000:0.0:1769074808.893736:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.893885:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074808.893973:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616973696 00000100:00100000:3.0:1769074808.893988:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616973696 00000100:00100000:3.0:1769074808.894038:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13419 00000100:00100000:3.0:1769074808.894061:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:64143:x1855007616973696:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074808.894086:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074808.894105:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54a99 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4b2 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.894206:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54aa0 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.894221:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54aa0 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:3.0:1769074808.894236:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54aa0 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:3.0:1769074808.894250:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x1fe:0x0]: returning size 2097152 00010000:00010000:3.0:1769074808.894351:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54a99 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4b2 expref: 16 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.894368:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54aa0 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4b2 expref: 15 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.894376:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074808.894379:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074808.894382:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba200, rc 0) 00000100:00100000:3.0:1769074808.894692:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:64143:x1855007616973696:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 644us (805us total) trans 8589944043 rc 0/0 00000100:00100000:3.0:1769074808.894711:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13419 00000100:00100000:1.0:1769074808.895860:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616973824 00000100:00100000:1.0:1769074808.895871:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616973824 00000100:00100000:0.0:1769074808.896321:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1038 00000100:00100000:0.0:1769074808.896332:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2406:x1855007616973824:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074808.896345:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074808.896366:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54a8b lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ab expref: 15 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074808.896383:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:0.0:1769074808.896387:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074808.896561:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007616973824:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 230us (2835us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.896569:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1038 00000100:00100000:0.0:1769074808.905769:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074808.905935:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616973952 00000100:00100000:2.0:1769074808.905945:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616973952 00000100:00100000:2.0:1769074808.906023:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8741 00000100:00100000:2.0:1769074808.906032:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2407:x1855007616973952:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074808.906075:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.906096:0:8404:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 5 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.906104:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074808.906107:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074808.906110:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 4/1,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.906144:0:8404:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 5 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.906158:0:8404:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 5 pid: 8398 timeout: 1896 lvb_type: 0 00000100:00100000:2.0:1769074808.906172:0:8404:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8404:1855007625206912:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074808.906255:0:8404:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074808.912112:0:8404:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8404:1855007625206912:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074808.912146:0:8404:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074808.915259:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074808.915434:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616974080 00000100:00100000:2.0:1769074808.915444:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616974080 00010000:00010000:2.0:1769074808.915486:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 6 pid: 8398 timeout: 1896 lvb_type: 0 00000100:00100000:2.0:1769074808.915524:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7187 00000100:00100000:2.0:1769074808.915531:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2409:x1855007616974080:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074808.915543:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074808.915559:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 6 pid: 8398 timeout: 1896 lvb_type: 0 00010000:00010000:2.0:1769074808.915568:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 6 pid: 8398 timeout: 1896 lvb_type: 0 00010000:00010000:2.0:1769074808.915586:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b682165400/0x7fe2323adfd54981 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c457 expref: 6 pid: 8398 timeout: 1896 lvb_type: 0 00010000:00010000:2.0:1769074808.915600:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074808.915601:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074808.915606:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.915615:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074808.915646:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074808.915697:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2409:x1855007616974080:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 166us (446us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074808.915714:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7187 00010000:00010000:3.0:1769074808.917203:0:8404:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.917214:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945484:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945504:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945519:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945526:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074808.945530:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945539:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945558:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074808.945565:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54aae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074808.945746:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2407:x1855007616973952:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 39708us (39981us total) trans 0 rc 2097152/2097152 00000100:00100000:3.0:1769074808.945757:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8741 00000100:00100000:0.0:1769074808.955498:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074808.955613:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616974208 00000100:00100000:1.0:1769074808.955625:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616974208 00000100:00100000:1.0:1769074808.955684:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1039 00000100:00100000:1.0:1769074808.955692:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:64143:x1855007616974208:12345-192.168.204.60@tcp:103:lfs.0 00010000:00010000:1.0:1769074808.955737:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074808.955766:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54aa0 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4b2 expref: 14 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074808.955786:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:1.0:1769074808.955791:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074808.956235:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:64143:x1855007616974208:12345-192.168.204.60@tcp:103:lfs.0 Request processed in 542us (747us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074808.956242:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1039 00000100:00100000:1.0:1769074808.958252:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074808.958342:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616974464 00000100:00100000:2.0:1769074808.958351:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616974464 00000100:00100000:2.0:1769074808.958432:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4710 00000100:00100000:2.0:1769074808.958443:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:64143:x1855007616974464:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074808.958685:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:64143:x1855007616974464:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 244us (439us total) trans 8589944044 rc 0/0 00000100:00100000:2.0:1769074808.958693:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4710 00000100:00100000:2.0:1769074808.961232:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074808.961311:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616974720 00000100:00100000:0.0:1769074808.961321:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616974720 00000100:00100000:0.0:1769074808.961372:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4711 00000100:00100000:0.0:1769074808.961383:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:64143:x1855007616974720:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074808.961631:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:64143:x1855007616974720:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 249us (405us total) trans 8589944045 rc 0/0 00000100:00100000:0.0:1769074808.961638:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4711 00010000:00010000:3.0:1769074809.247892:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1769074809.253853:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874608 is committed 00000100:00100000:3.0:1769074809.757657:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074809.757679:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625207040:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074809.757770:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074809.761707:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207040 00000100:00100000:2.0:1769074809.761723:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207040 00000100:00100000:2.0:1769074809.761830:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2677 00000100:00100000:2.0:1769074809.761842:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625207040:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074809.761882:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00100000:2.0:1769074809.761973:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625207040:12345-0@lo:41:osp-pre-0-1.0 Request processed in 134us (4203us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074809.761992:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2677 00000100:00100000:3.0:1769074809.762183:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625207040:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074809.762847:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074809.762863:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625207168:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074809.762911:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074809.763242:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207168 00000100:00100000:0.0:1769074809.763249:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207168 00000100:00100000:0.0:1769074809.763298:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2678 00000100:00100000:0.0:1769074809.763307:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625207168:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074809.763333:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:0.0:1769074809.763382:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625207168:12345-0@lo:41:osp-pre-1-0.0 Request processed in 76us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074809.763389:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2678 00000100:00100000:0.0:1769074809.763427:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625207168:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074810.269073:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625207424:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074810.269091:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625207296:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1769074810.271144:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074810.274162:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074810.274240:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074810.274254:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625207552:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074810.274289:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074810.274349:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207424 00000100:00100000:2.0:1769074810.274362:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207424 00000100:00100000:2.0:1769074810.274560:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207296 00000100:00100000:2.0:1769074810.274564:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207296 00000100:00100000:2.0:1769074810.274593:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1196 00000100:00100000:2.0:1769074810.274603:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625207424:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074810.274644:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 499651904 pending 0 free 3911847936 avail 3680325632 00000100:00100000:0.0:1769074810.280312:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1197 00000100:00100000:0.0:1769074810.280341:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625207296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074810.280377:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:0.0:1769074810.280471:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625207296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 146us (6188us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074810.280479:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1197 00000100:00080000:0.0:1769074810.280519:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074810.280526:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074810.280552:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874608 gen 2 00000004:00080000:1.0:1769074810.280653:0:3652:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 17179874607 committed transno 17179874608 00000004:00080000:1.0:1769074810.280699:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546f91180, transno 17179874608 00000100:00100000:1.0:1769074810.280737:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625207296:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074810.284446:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625207424:12345-0@lo:13:osp-pre-0-0.0 Request processed in 9829us (10275us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074810.284468:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1196 00000100:00100000:3.0:1769074810.287168:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074810.287249:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625207424:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074810.289609:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207552 00000100:00100000:1.0:1769074810.289630:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207552 00000100:00100000:1.0:1769074810.289754:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1198 00000100:00100000:1.0:1769074810.289765:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625207552:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074810.289801:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 499651904 pending 0 free 3911847936 avail 3680325632 00000100:00100000:1.0:1769074810.289928:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625207552:12345-0@lo:13:osp-pre-0-1.0 Request processed in 164us (2766us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074810.289938:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1198 00000100:00080000:3.0:1769074810.290794:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074810.291001:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625207552:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074810.788374:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074810.788401:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625207680:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074810.788476:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074810.788546:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207680 00000100:00100000:3.0:1769074810.788556:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207680 00000100:00100000:3.0:1769074810.788619:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1199 00000100:00100000:3.0:1769074810.788627:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625207680:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074810.788651:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:3.0:1769074810.788708:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625207680:12345-0@lo:13:osp-pre-1-1.0 Request processed in 81us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074810.788715:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1199 00000100:00080000:3.0:1769074810.788739:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074810.788776:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625207680:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074811.292970:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074812.318058:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1769074812.336879:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944045 is committed 00000020:00080000:0.0:1769074812.336903:0:32377:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 4294969364 batchid updated 00000020:00080000:3.0:1769074812.336968:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969364 00000020:00080000:3.0:1769074812.337098:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294969364 cancel update log [0x200000bd0:0x2:0x0].82: rc = 0 00000020:00080000:3.0:1769074812.337111:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969364 00000100:00100000:2.0:1769074812.829250:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625208064:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829319:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074812.829377:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625208576:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829398:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074812.829424:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625209088:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829450:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074812.829478:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1769074812.829485:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625207936:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829509:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074812.829535:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625208448:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829558:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074812.829581:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625208960:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829602:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074812.829644:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208064 00000100:00100000:2.0:1769074812.829651:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208064 00000100:00100000:2.0:1769074812.829705:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208576 00000100:00100000:2.0:1769074812.829709:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208576 00000100:00100000:2.0:1769074812.829734:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208448 00000100:00100000:2.0:1769074812.829737:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208448 00000100:00100000:2.0:1769074812.829751:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2382 00000100:00100000:2.0:1769074812.829759:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625208064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829840:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625208064:12345-0@lo:400:kworker.0 Request processed in 81us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074812.829851:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2382 00000100:00100000:2.0:1769074812.829870:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2383 00000100:00100000:2.0:1769074812.829876:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625208576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829904:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625208576:12345-0@lo:400:kworker.0 Request processed in 29us (504us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074812.829909:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2383 00000100:00100000:2.0:1769074812.829920:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2384 00000100:00100000:2.0:1769074812.829926:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625208448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.829951:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625208448:12345-0@lo:400:kworker.0 Request processed in 25us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074812.829957:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2384 00000100:00100000:2.0:1769074812.829979:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209088 00000100:00100000:2.0:1769074812.829983:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209088 00000100:00100000:2.0:1769074812.830010:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1494 00000100:00100000:2.0:1769074812.830017:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625209088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830046:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625209088:12345-0@lo:400:kworker.0 Request processed in 30us (595us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074812.830052:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1494 00000100:00100000:2.0:1769074812.830068:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207936 00000100:00100000:2.0:1769074812.830071:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207936 00000100:00100000:2.0:1769074812.830091:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208960 00000100:00100000:2.0:1769074812.830094:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208960 00000100:00100000:2.0:1769074812.830109:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2679 00000100:00100000:2.0:1769074812.830115:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625207936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830145:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625207936:12345-0@lo:400:kworker.0 Request processed in 30us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074812.830153:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2679 00000100:00100000:2.0:1769074812.830166:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2680 00000100:00100000:2.0:1769074812.830172:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625208960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830197:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625208960:12345-0@lo:400:kworker.0 Request processed in 26us (594us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074812.830202:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2680 00000100:00080000:2.0:1769074812.830221:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074812.830225:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074812.830227:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074812.830258:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625207936:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830268:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625208448:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830275:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625208960:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830283:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625208064:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830291:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625208576:0@lo:400:kworker.0 00000100:00100000:2.0:1769074812.830299:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625209088:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831324:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625208192:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831377:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831435:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625208704:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831461:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831476:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625209216:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831495:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831520:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:1.0:1769074812.831527:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625207808:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831551:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831572:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625208320:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831592:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831609:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625208832:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831628:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831654:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625209344:0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831687:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074812.831731:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625207808 00000100:00100000:1.0:1769074812.831736:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625207808 00000100:00100000:1.0:1769074812.831775:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:1.0:1769074812.831783:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681597050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625207808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074812.831845:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681597050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625207808:12345-0@lo:400:kworker.0 Request processed in 63us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074812.831852:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:1.0:1769074812.831885:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208192 00000100:00100000:1.0:1769074812.831889:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208192 00000100:00100000:1.0:1769074812.831919:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209216 00000100:00100000:1.0:1769074812.831922:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209216 00000100:00100000:1.0:1769074812.831939:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208320 00000100:00100000:1.0:1769074812.831942:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208320 00000100:00100000:0.0:1769074812.832294:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1495 00000100:00100000:0.0:1769074812.832305:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625208192:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832345:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625208192:12345-0@lo:400:kworker.0 Request processed in 43us (967us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074812.832350:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1495 00000100:00100000:0.0:1769074812.832366:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1496 00000100:00100000:0.0:1769074812.832372:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625209216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832398:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625209216:12345-0@lo:400:kworker.0 Request processed in 27us (901us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074812.832404:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1496 00000100:00100000:0.0:1769074812.832417:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1497 00000100:00100000:0.0:1769074812.832422:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625208320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832447:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625208320:12345-0@lo:400:kworker.0 Request processed in 25us (854us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074812.832451:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1497 00000100:00100000:0.0:1769074812.832474:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208704 00000100:00100000:0.0:1769074812.832477:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208704 00000100:00100000:0.0:1769074812.832505:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625208832 00000100:00100000:0.0:1769074812.832508:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625208832 00000100:00100000:0.0:1769074812.832523:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209344 00000100:00100000:0.0:1769074812.832525:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209344 00000100:00100000:0.0:1769074812.832536:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2385 00000100:00100000:0.0:1769074812.832542:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625208704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832570:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625208704:12345-0@lo:400:kworker.0 Request processed in 28us (1109us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074812.832575:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2385 00000100:00100000:0.0:1769074812.832587:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2386 00000100:00100000:0.0:1769074812.832593:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625208832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832619:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625208832:12345-0@lo:400:kworker.0 Request processed in 27us (990us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074812.832625:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2386 00000100:00100000:0.0:1769074812.832644:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2387 00000100:00100000:0.0:1769074812.832650:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625209344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832683:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625209344:12345-0@lo:400:kworker.0 Request processed in 33us (996us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074812.832691:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2387 00000100:00100000:0.0:1769074812.832746:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625207808:0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832757:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625208320:0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832766:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625208832:0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832775:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625209344:0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832784:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625208192:0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832793:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625208704:0@lo:400:kworker.0 00000100:00100000:0.0:1769074812.832801:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625209216:0@lo:400:kworker.0 00000100:00080000:0.0:1769074812.832863:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074812.832867:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074812.832869:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074812.832871:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1769074813.341727:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074813.892552:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074813.892604:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074813.892647:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074813.892673:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074813.892702:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074813.892731:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975104 00000100:00100000:0.0:1769074813.892734:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074813.892759:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074813.892762:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975104 00000100:00100000:0.0:1769074813.892784:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074813.892878:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074813.892912:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975616 00000100:00100000:3.0:1769074813.892918:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975616 00000100:00100000:0.0:1769074813.892927:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975360 00000100:00100000:0.0:1769074813.892940:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975360 00000100:00100000:3.0:1769074813.892970:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616974976 00000100:00100000:3.0:1769074813.892977:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616974976 00000100:00100000:0.0:1769074813.893004:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975872 00000100:00100000:3.0:1769074813.893010:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975488 00000100:00100000:0.0:1769074813.893011:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975872 00000100:00100000:3.0:1769074813.893018:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975488 00000100:00100000:3.0:1769074813.893055:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2388 00000100:00100000:0.0:1769074813.893059:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975232 00000100:00100000:0.0:1769074813.893066:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975232 00000100:00100000:3.0:1769074813.893074:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616975104:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074813.893094:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616975744 00000100:00100000:0.0:1769074813.893100:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616975744 00000100:00100000:0.0:1769074813.893131:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1498 00000100:00100000:0.0:1769074813.893142:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616975360:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074813.893200:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616975104:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 129us (589us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074813.893211:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2388 00000100:00100000:0.0:1769074813.893222:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616975360:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 81us (572us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074813.893247:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2389 00000100:00100000:0.0:1769074813.893249:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1498 00000100:00100000:3.0:1769074813.893267:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616975616:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074813.893284:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1499 00000100:00100000:0.0:1769074813.893298:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616975872:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074813.893320:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616975616:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 57us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074813.893331:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2389 00000100:00100000:0.0:1769074813.893347:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2408:x1855007616975872:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 50us (641us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074813.893356:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1499 00000100:00100000:3.0:1769074813.893373:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2390 00000100:00100000:0.0:1769074813.893380:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1500 00000100:00100000:3.0:1769074813.893392:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007616974976:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074813.893396:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616975232:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074813.893439:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616975232:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 49us (678us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074813.893446:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1500 00000100:00100000:3.0:1769074813.893453:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007616974976:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 67us (717us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074813.893463:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1501 00000100:00100000:3.0:1769074813.893465:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2390 00000100:00100000:0.0:1769074813.893481:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616975744:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074813.893493:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2391 00000100:00100000:3.0:1769074813.893508:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616975488:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074813.893527:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616975744:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 49us (646us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074813.893535:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1501 00000100:00100000:3.0:1769074813.893569:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2408:x1855007616975488:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 64us (783us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074813.893578:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2391 00000100:00080000:0.0:1769074813.893594:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074813.893600:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074813.893606:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074813.893610:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074813.893776:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616974848 00000100:00100000:2.0:1769074813.893782:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616974848 00000100:00100000:2.0:1769074813.893855:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 359 00000100:00100000:2.0:1769074813.893868:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa44450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2408:x1855007616974848:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074813.893918:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa44450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2408:x1855007616974848:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 56us (1377us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074813.893925:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 359 00010000:00010000:2.0:1769074814.366239:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074814.756423:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074814.756536:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616976128 00000100:00100000:3.0:1769074814.756546:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616976128 00000100:00100000:3.0:1769074814.756609:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13420 00000100:00100000:3.0:1769074814.756621:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64571:x1855007616976128:12345-192.168.204.60@tcp:36:rm.0 00010000:00010000:3.0:1769074814.756674:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 4 locks, starting at 0 00010000:00010000:3.0:1769074814.756699:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54a76 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c49d expref: 10 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074814.756722:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd549ea lrc: 0/0,0 mode: --/PR res: [0x200000405:0x1fe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c481 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074814.756734:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd549dc lrc: 0/0,0 mode: --/CR res: [0x200000405:0x1fe:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c47a expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074814.756745:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad800) 00010000:00010000:3.0:1769074814.756760:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54a06 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c488 expref: 7 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074814.756769:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074814.756772:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074814.756794:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756836:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756847:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756861:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756873:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756879:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756966:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756975:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.756982:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000040:00080000:3.0:1769074814.757210:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2461, 64 off210304 00000004:00080000:3.0:1769074814.757221:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1802599413310ns, next at 1803599410455ns, rc = 0 00000040:00080000:3.0:1769074814.757232:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2462, 64 off210368 00000040:00080000:3.0:1769074814.757253:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2466, 64 off210984 00000004:00080000:3.0:1769074814.757259:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1802599451968ns, next at 1803599450775ns, rc = 0 00010000:00010000:3.0:1769074814.757307:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 3/0,1 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757316:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757324:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757331:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757338:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757345:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757366:0:6512:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074814.757385:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757392:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757398:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757403:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074814.757408:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757414:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.757422:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad600) 00010000:00010000:3.0:1769074814.757426:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1769074814.757429:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074814.758535:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074814.758544:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074814.758553:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074814.758556:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074814.758563:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074814.758575:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x1fe:0x0].0x0 (ffff89b6439ad600) 00010000:00010000:1.0:1769074814.758578:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1769074814.758582:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 1/0,0 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074814.758587:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54aca lrc: 0/0,0 mode: --/PW res: [0x200000405:0x1fe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074814.758599:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074814.758609:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074814.758612:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074814.761571:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54ac3 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x1fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.761770:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.761778:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.761787:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.761794:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.761799:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074814.763724:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.763748:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.763767:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074814.763770:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074814.763776:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.763783:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54ab5 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000004:00010000:3.0:1769074814.763800:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000003d238b13 reply state 0000000001be3ce1 transno 8589944046 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.764491:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074814.764498:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b6439adf00) 00010000:00010000:3.0:1769074814.764500:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074814.764504:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074814.764620:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:64571:x1855007616976128:12345-192.168.204.60@tcp:36:rm.0 Request processed in 7996us (8201us total) trans 8589944046 rc 0/0 00000100:00100000:3.0:1769074814.764634:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13420 00000100:00100000:2.0:1769074814.880209:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074814.880232:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625209472:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074814.880299:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074814.880346:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209472 00000100:00100000:2.0:1769074814.880354:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209472 00000100:00100000:2.0:1769074814.880402:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2681 00000100:00100000:2.0:1769074814.880410:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625209472:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074814.880446:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1445404672 avail 1299165184 00000100:00100000:2.0:1769074814.880491:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625209472:12345-0@lo:41:osp-pre-0-1.0 Request processed in 81us (190us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074814.880498:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2681 00000100:00100000:2.0:1769074814.880546:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625209472:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074814.884996:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074814.885020:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625209600:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074814.885092:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074814.885139:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209600 00000100:00100000:0.0:1769074814.885148:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209600 00000100:00100000:0.0:1769074814.885202:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2682 00000100:00100000:0.0:1769074814.885219:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625209600:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074814.885255:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446150144 avail 1299910656 00000100:00100000:0.0:1769074814.885304:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625209600:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074814.885311:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2682 00000100:00100000:0.0:1769074814.885354:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625209600:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074815.076333:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074815.076577:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616976256 00000100:00100000:0.0:1769074815.076591:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616976256 00000100:00100000:0.0:1769074815.076768:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13421 00000100:00100000:0.0:1769074815.076781:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:64582:x1855007616976256:12345-192.168.204.60@tcp:101:lsof.0 00010000:00010000:0.0:1769074815.076864:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074815.076896:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54ad1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4b9 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074815.076999:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54ad8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074815.077028:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074815.077042:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:0.0:1769074815.077107:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1769074815.077113:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074815.077160:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074815.077190:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54ad1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4b9 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074815.077214:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54ad8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4b9 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074815.077225:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074815.077228:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769074815.077232:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164600, rc 0) 00000100:00100000:0.0:1769074815.077315:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64582:x1855007616976256:12345-192.168.204.60@tcp:101:lsof.0 Request processed in 537us (993us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074815.077324:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13421 00000100:00100000:0.0:1769074815.087899:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074815.088038:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616976384 00000100:00100000:3.0:1769074815.088051:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616976384 00000100:00080000:2.0:1769074815.088207:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074815.103861:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13422 00000100:00100000:3.0:1769074815.103885:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:64582:x1855007616976384:12345-192.168.204.60@tcp:101:lsof.0 00010000:00010000:2.0:1769074815.103977:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074815.104010:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54adf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4c0 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074815.104105:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54ae6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074815.104124:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074815.104139:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074815.104383:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074815.104387:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074815.104416:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074815.104448:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54adf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4c0 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074815.104471:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54ae6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4c0 expref: 6 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074815.104481:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074815.104485:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074815.104489:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88000, rc 0) 00000100:00100000:2.0:1769074815.106362:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:64582:x1855007616976384:12345-192.168.204.60@tcp:101:lsof.0 Request processed in 2472us (18464us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074815.106379:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13422 00000100:00100000:0.0:1769074815.389257:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625209728:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074815.389367:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074815.389485:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209728 00000100:00100000:0.0:1769074815.389496:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209728 00000100:00100000:0.0:1769074815.389579:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1200 00000100:00100000:0.0:1769074815.389603:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625209728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074815.389642:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 499651904 pending 0 free 3911847936 avail 3680325632 00000100:00100000:0.0:1769074815.389696:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625209728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 108us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074815.389704:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1200 00000100:00100000:0.0:1769074815.389943:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625209728:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1769074815.389972:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074815.390342:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074815.390353:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625209856:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074815.390392:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074815.390430:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209856 00000100:00100000:0.0:1769074815.390435:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209856 00000100:00100000:0.0:1769074815.390469:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1201 00000100:00100000:0.0:1769074815.390477:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625209856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074815.390497:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:0.0:1769074815.390533:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625209856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 56us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074815.390540:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1201 00000100:00100000:0.0:1769074815.390576:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625209856:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074815.391579:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625209984:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074815.391654:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1769074815.391714:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1769074815.391770:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074815.392170:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625209984 00000100:00100000:3.0:1769074815.392181:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625209984 00000100:00100000:3.0:1769074815.392258:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1202 00000100:00100000:3.0:1769074815.392269:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625209984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074815.392293:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 499651904 pending 0 free 3911847936 avail 3680325632 00000100:00100000:3.0:1769074815.392356:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625209984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 90us (702us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074815.392364:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1202 00000100:00100000:3.0:1769074815.392420:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625209984:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1769074815.395047:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074815.901757:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625210112:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074815.902479:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074815.905039:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210112 00000100:00100000:3.0:1769074815.905062:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210112 00000100:00100000:3.0:1769074815.905190:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1203 00000100:00100000:3.0:1769074815.905203:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625210112:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074815.905239:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 493335872 pending 0 free 3917004800 avail 3685482496 00000100:00100000:3.0:1769074815.905354:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625210112:12345-0@lo:13:osp-pre-1-1.0 Request processed in 153us (2876us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074815.905363:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1203 00000100:00100000:2.0:1769074815.906234:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625210112:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1769074815.906408:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074816.414872:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074817.440738:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074817.950936:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625210496:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951038:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.951116:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625211008:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951140:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.951156:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625211520:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951180:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.951211:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1769074817.951218:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625210624:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951238:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.951262:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625211136:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951286:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.951303:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625211648:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951324:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.951355:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211520 00000100:00100000:3.0:1769074817.951364:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211520 00000100:00100000:3.0:1769074817.951425:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210624 00000100:00100000:3.0:1769074817.951428:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210624 00000100:00100000:3.0:1769074817.951448:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211648 00000100:00100000:3.0:1769074817.951450:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211648 00000100:00100000:3.0:1769074817.951473:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1502 00000100:00100000:3.0:1769074817.951481:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625211520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951572:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625211520:12345-0@lo:400:kworker.0 Request processed in 91us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.951581:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1502 00000100:00100000:3.0:1769074817.951600:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1503 00000100:00100000:3.0:1769074817.951606:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625210624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951633:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625210624:12345-0@lo:400:kworker.0 Request processed in 28us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.951639:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1503 00000100:00100000:3.0:1769074817.951649:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1504 00000100:00100000:3.0:1769074817.951655:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625211648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951684:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625211648:12345-0@lo:400:kworker.0 Request processed in 29us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.951690:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1504 00000100:00100000:3.0:1769074817.951752:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625210624:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.951765:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625211648:0@lo:400:kworker.0 00000100:00100000:0.0:1769074817.951772:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210496 00000100:00100000:3.0:1769074817.951775:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625211520:0@lo:400:kworker.0 00000100:00100000:0.0:1769074817.951777:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210496 00000100:00080000:3.0:1769074817.951825:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074817.951831:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074817.951846:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211008 00000100:00100000:0.0:1769074817.951849:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211008 00000100:00100000:3.0:1769074817.951861:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211136 00000100:00100000:3.0:1769074817.951865:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211136 00000100:00100000:2.0:1769074817.951931:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625210368:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.951974:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952000:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625210880:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952024:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952042:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625211392:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952066:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952089:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:2.0:1769074817.952095:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625210240:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952125:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952150:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625210752:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952173:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952192:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625211264:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952214:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952225:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625211776:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952245:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074817.952268:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210368 00000100:00100000:2.0:1769074817.952272:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210368 00000100:00100000:0.0:1769074817.952279:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210880 00000100:00100000:0.0:1769074817.952282:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210880 00000100:00100000:2.0:1769074817.952307:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211392 00000100:00100000:2.0:1769074817.952311:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211392 00000100:00100000:2.0:1769074817.952333:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2683 00000100:00100000:2.0:1769074817.952339:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625210368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952380:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625210368:12345-0@lo:400:kworker.0 Request processed in 41us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074817.952386:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2683 00000100:00100000:2.0:1769074817.952402:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2684 00000100:00100000:2.0:1769074817.952407:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625211392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952443:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625211392:12345-0@lo:400:kworker.0 Request processed in 35us (376us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074817.952448:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2684 00000100:00100000:2.0:1769074817.952471:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210240 00000100:00100000:2.0:1769074817.952474:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210240 00000100:00100000:2.0:1769074817.952501:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1769074817.952507:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa46050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625210240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.952564:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa46050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625210240:12345-0@lo:400:kworker.0 Request processed in 57us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074817.952571:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1769074817.952613:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625210752 00000100:00100000:2.0:1769074817.952617:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625210752 00000100:00100000:1.0:1769074817.953053:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211264 00000100:00100000:1.0:1769074817.953060:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211264 00000100:00100000:3.0:1769074817.954526:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211776 00000100:00100000:3.0:1769074817.954542:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211776 00000100:00100000:3.0:1769074817.954577:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2392 00000100:00100000:3.0:1769074817.954587:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625210496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.954705:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625210496:12345-0@lo:400:kworker.0 Request processed in 119us (3667us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.954714:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2392 00000100:00100000:3.0:1769074817.954735:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2393 00000100:00100000:3.0:1769074817.954743:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625211008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.954778:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625211008:12345-0@lo:400:kworker.0 Request processed in 37us (3637us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.954784:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2393 00000100:00100000:3.0:1769074817.954796:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2394 00000100:00100000:3.0:1769074817.954849:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625211136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.954895:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625211136:12345-0@lo:400:kworker.0 Request processed in 93us (3607us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.954902:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2394 00000100:00100000:3.0:1769074817.954921:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2395 00000100:00100000:3.0:1769074817.954927:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625211776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.954974:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625211776:12345-0@lo:400:kworker.0 Request processed in 47us (2727us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074817.954980:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2395 00000100:00100000:3.0:1769074817.955069:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625211136:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.955092:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625210496:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.955104:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625211008:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.955212:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625210240:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.955226:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625211776:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.955238:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625210368:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.955251:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625211392:0@lo:400:kworker.0 00000100:00080000:3.0:1769074817.955299:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074817.955307:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074817.955310:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074817.959616:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2396 00000100:00100000:2.0:1769074817.959791:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1505 00000100:00100000:2.0:1769074817.959801:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625210752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.960020:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625210752:12345-0@lo:400:kworker.0 Request processed in 219us (7844us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074817.960029:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1505 00000100:00100000:0.0:1769074817.960128:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625210880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074817.960176:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625210880:12345-0@lo:400:kworker.0 Request processed in 546us (8150us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074817.960183:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2396 00000100:00100000:3.0:1769074817.961143:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625210752:0@lo:400:kworker.0 00000100:00100000:3.0:1769074817.961169:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625210880:0@lo:400:kworker.0 00000100:00100000:2.0:1769074817.961397:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074817.961447:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074817.961577:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074817.961625:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616976512 00000100:00100000:2.0:1769074817.961634:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616976512 00000100:00100000:2.0:1769074817.961709:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616976768 00000100:00100000:2.0:1769074817.961713:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616976768 00000100:00100000:2.0:1769074817.961742:0:33102:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616976640 00000100:00100000:2.0:1769074817.961745:0:33102:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616976640 00000100:00100000:2.0:1769074817.961782:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1040 00000100:00100000:2.0:1769074817.961798:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a7835f93-1e5d-43a4-8110-91f072d9ae65+14:2409:x1855007616976512:12345-192.168.204.60@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1769074817.962136:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1041 00000100:00100000:1.0:1769074817.962152:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a7835f93-1e5d-43a4-8110-91f072d9ae65+14:2409:x1855007616976768:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074817.962170:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074817.962194:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b67b446a00/0x7fe2323adfd5479e lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 8 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3a8 expref: 14 pid: 32381 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074817.962216:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074817.962279:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a7835f93-1e5d-43a4-8110-91f072d9ae65+13:2409:x1855007616976768:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 129us (831us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074817.962288:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1041 00000100:00100000:1.0:1769074817.962310:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1042 00000100:00100000:1.0:1769074817.962317:0:6499:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a7835f93-1e5d-43a4-8110-91f072d9ae65+12:2409:x1855007616976640:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074817.962325:0:6499:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074817.962334:0:6499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b666b89c00/0x7fe2323adfd54797 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 6 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c3a1 expref: 12 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074817.962343:0:6499:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074817.962361:0:6499:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a7835f93-1e5d-43a4-8110-91f072d9ae65+11:2409:x1855007616976640:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 45us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074817.962367:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1042 00010000:00010000:2.0:1769074817.962612:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074817.962625:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff89b547866000/0x7fe2323adfd54790 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c370 expref: 10 pid: 32382 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.962634:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074817.962656:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007616976512:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 860us (1275us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074817.962663:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1040 00000100:00100000:0.0:1769074817.963093:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2397 00000100:00100000:0.0:1769074817.963105:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625211264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074817.963183:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625211264:12345-0@lo:400:kworker.0 Request processed in 81us (10968us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074817.963190:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2397 00000001:00080000:2.0:1769074817.963714:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944046 is committed 00000004:00080000:2.0:1769074817.963724:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1962 committed 00000004:00080000:2.0:1769074817.963733:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1959 committed 00000100:00100000:2.0:1769074817.963829:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625211264:0@lo:400:kworker.0 00010000:00010000:0.0:1769074817.963913:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074817.963925:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074817.963932:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074817.963939:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074817.963944:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074817.963952:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074817.963978:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa976bffe (ffff89b6439adf00) 00010000:00010000:0.0:1769074817.963982:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074817.963987:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074817.963993:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54abc lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa976bffe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:2.0:1769074817.965803:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625211904:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1769074817.965969:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074817.967761:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625211904 00000100:00100000:0.0:1769074817.967780:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625211904 00000100:00100000:0.0:1769074817.968021:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14445 00000100:00100000:0.0:1769074817.968034:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625211904:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1769074817.968075:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:709 count 1 00010000:00010000:0.0:1769074817.968109:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.968136:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 5 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.968146:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074817.968149:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074817.968154:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.968210:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 5 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.968227:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 5 pid: 9048 timeout: 1905 lvb_type: 0 00000100:00100000:0.0:1769074817.968245:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa20e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625212032:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074817.968372:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b647293b40 going to sleep for 11 seconds 00000100:00100000:1.0:1769074817.976410:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074817.976447:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625212160:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1769074817.976575:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074817.976645:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625212160 00000100:00100000:3.0:1769074817.976657:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625212160 00000100:00100000:3.0:1769074817.976731:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14446 00000100:00100000:3.0:1769074817.976743:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625212160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1769074817.976777:0:28955:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:705 count 1 00010000:00010000:3.0:1769074817.976842:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074817.976870:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 5 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074817.976890:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074817.976893:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074817.976898:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074817.976934:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 5 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074817.976947:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 5 pid: 8396 timeout: 1905 lvb_type: 1 00000100:00100000:3.0:1769074817.976959:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625212288:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074817.977041:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5410b8240 going to sleep for 11 seconds 00000100:00100000:2.0:1769074817.987520:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074817.987631:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977152 00000100:00100000:1.0:1769074817.987645:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977152 00000100:00100000:3.0:1769074817.987684:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625212288:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074817.987724:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074817.987725:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 6 pid: 8396 timeout: 1905 lvb_type: 1 00000100:00100000:1.0:1769074817.987782:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7188 00000100:00100000:1.0:1769074817.987795:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2406:x1855007616977152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074817.987870:0:27843:0:(ldlm_lockd.c:740:ldlm_handle_ast_error()) ### client (nid 192.168.204.60@tcp) returned -22 from blocking AST (req@00000000c275f6de x1855007625212032) - normal race ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 5 pid: 9048 timeout: 1905 lvb_type: 0 00010000:00010000:0.0:1769074817.987929:0:27843:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 5 pid: 9048 timeout: 1905 lvb_type: 0 00010000:00010000:1.0:1769074817.987953:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074817.987962:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b547866200/0x7fe2323adfd54988 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c45e expref: 5 pid: 9048 timeout: 1905 lvb_type: 0 00010000:00010000:1.0:1769074817.987966:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 6 pid: 8396 timeout: 1905 lvb_type: 1 00010000:00010000:1.0:1769074817.987979:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 6 pid: 8396 timeout: 1905 lvb_type: 1 00010000:00010000:1.0:1769074817.987996:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54a7d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4a4 expref: 6 pid: 8396 timeout: 1905 lvb_type: 1 00010000:00010000:1.0:1769074817.988026:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074817.988029:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074817.988034:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074817.988045:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074817.988067:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074817.988138:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2406:x1855007616977152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 344us (634us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074817.988147:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7188 00000100:00100000:0.0:1769074817.988211:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625212032:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074817.988228:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074817.988230:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074817.988235:0:27843:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988244:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988260:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988269:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988278:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988289:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988298:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074817.988319:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074817.988694:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074817.988706:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074817.988715:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074817.988723:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074817.988740:0:28955:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1769074817.989355:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625211904:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21321us (23383us total) trans 17179874609 rc 0/0 00000100:00100000:1.0:1769074817.989367:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14445 00010000:00010000:2.0:1769074817.989644:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1769074817.989656:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989666:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074817.989671:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989682:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989703:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989711:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b682164a00/0x7fe2323adfd54aed lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989737:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074817.989741:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989747:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074817.989750:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989756:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989766:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989772:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54af4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074817.989785:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074817.989786:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074817.989788:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1769074817.989801:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074817.990123:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000004:00080000:1.0:1769074817.990160:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67fa20700/2, rc 0, transno 17179874609 00000100:00100000:0.0:1769074817.990171:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977024 00000100:00100000:1.0:1769074817.990172:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625211904:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1769074817.990179:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977024 00000100:00100000:1.0:1769074817.990245:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14447 00010000:00010000:2.0:1769074817.990282:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1769074817.990289:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074817.990324:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2405:x1855007616977024:12345-192.168.204.60@tcp:9:umount.0 00000020:00000080:1.0:1769074817.990363:0:27844:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd547ba 00000100:00080000:1.0:1769074817.990373:0:27844:0:(import.c:65:import_set_state_nolock()) ffff89b66e781000 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1769074817.990478:0:27844:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 4, last_transno is 17179874609 00000100:00100000:1.0:1769074817.990647:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+2:2405:x1855007616977024:12345-192.168.204.60@tcp:9:umount.0 Request processed in 391us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074817.994225:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14447 00010000:00010000:2.0:1769074817.994355:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074817.996101:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625212160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19350us (19515us total) trans 21474841282 rc 0/0 00000100:00100000:3.0:1769074817.996118:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14446 00000004:00080000:1.0:1769074817.996190:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546c68700/2, rc 0, transno 21474841282 00000100:00100000:1.0:1769074817.996218:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625212160:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1769074818.017932:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074818.018203:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977280 00000100:00100000:3.0:1769074818.018218:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977280 00000100:00100000:3.0:1769074818.018304:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14448 00000100:00080000:2.0:1769074818.018391:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074818.018419:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2405:x1855007616977280:12345-192.168.204.60@tcp:9:umount.0 00000020:00000080:3.0:1769074818.018460:0:28955:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd547b3 00000100:00080000:3.0:1769074818.018472:0:28955:0:(import.c:65:import_set_state_nolock()) ffff89b649251800 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1769074818.018538:0:28955:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 5, last_transno is 21474841282 00010000:00010000:2.0:1769074818.018680:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074818.019085:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+2:2405:x1855007616977280:12345-192.168.204.60@tcp:9:umount.0 Request processed in 770us (1166us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074818.019094:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14448 00000100:00100000:0.0:1769074818.039147:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074818.039296:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977536 00000100:00100000:2.0:1769074818.039311:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977536 00000100:00100000:2.0:1769074818.039412:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13423 00000100:00100000:2.0:1769074818.039425:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+6:2405:x1855007616977536:12345-192.168.204.60@tcp:39:umount.0 00000020:00000080:2.0:1769074818.039485:0:6514:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd547a5 00000020:00010000:2.0:1769074818.039496:0:6514:0:(genops.c:1938:obd_stale_export_put()) Put export ffff89b543683000: total 0 00000100:00080000:2.0:1769074818.039502:0:6514:0:(import.c:65:import_set_state_nolock()) ffff89b643aae800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1769074818.039776:0:6514:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 2, last_transno is 8589944046 00000100:00100000:2.0:1769074818.040045:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2405:x1855007616977536:12345-192.168.204.60@tcp:39:umount.0 Request processed in 622us (913us total) trans 8589944047 rc 0/0 00000100:00100000:2.0:1769074818.040055:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13423 00000100:00080000:3.0:1769074818.040406:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000020:00010000:1.0:1769074818.043488:0:6502:0:(genops.c:1924:obd_stale_export_get()) Get export ffff89b543683000: total 0 00010000:00010000:1.0:1769074818.043502:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074818.043612:0:6502:0:(ldlm_lock.c:2645:ldlm_export_cancel_blocked_locks()) Export ffff89b543683000, canceled 0 locks, left on hash table 1. 00010000:00010000:1.0:1769074818.043665:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074818.043670:0:6502:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1769074818.043686:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54ae6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4c0 expref: 4 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074818.043715:0:6502:0:(ldlm_lock.c:2673:ldlm_export_cancel_locks()) Export ffff89b543683000, canceled 1 locks, left on hash table 0. 00010000:00010000:1.0:1769074818.043727:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074818.043729:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074818.043731:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074818.049472:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074818.049613:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977664 00000100:00100000:1.0:1769074818.049627:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977664 00000100:00100000:1.0:1769074818.049720:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13424 00000100:00100000:1.0:1769074818.049733:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+5:2405:x1855007616977664:12345-192.168.204.60@tcp:39:umount.0 00000020:00000080:1.0:1769074818.049780:0:6512:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd547ac 00000100:00080000:1.0:1769074818.049791:0:6512:0:(import.c:65:import_set_state_nolock()) ffff89b54361a000 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1769074818.049852:0:6512:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-MDT0001_UUID: mount_count is 1, last_transno is 4294968348 00000100:00100000:1.0:1769074818.050071:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2274e3cf-6162-4197-bb1e-d97abc9b4bb9+3:2405:x1855007616977664:12345-192.168.204.60@tcp:39:umount.0 Request processed in 341us (615us total) trans 4294968349 rc 0/0 00000100:00100000:1.0:1769074818.050079:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13424 00010000:00010000:0.0:1769074818.050157:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1769074818.050592:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1769074818.461177:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074819.012944:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074819.013007:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074819.013052:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074819.013075:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074819.013107:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074819.013142:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977792 00000100:00100000:2.0:1769074819.013154:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977792 00000100:00100000:2.0:1769074819.013218:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 361 00000100:00100000:2.0:1769074819.013229:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545175850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007616977792:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074819.013319:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545175850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007616977792:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 92us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074819.013327:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 361 00000100:00100000:1.0:1769074819.013524:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616978176 00000100:00100000:1.0:1769074819.013566:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616978176 00000100:00100000:1.0:1769074819.013617:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616978304 00000100:00100000:1.0:1769074819.013620:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616978304 00000100:00100000:1.0:1769074819.013642:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1506 00000100:00100000:1.0:1769074819.013648:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616978176:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074819.013687:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616978176:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 39us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074819.013694:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1506 00000100:00100000:1.0:1769074819.013709:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14449 00000100:00100000:1.0:1769074819.013716:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2405:x1855007616978304:12345-192.168.204.60@tcp:9:kworker.0 00000020:00000080:1.0:1769074819.013743:0:27844:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x7fe2323adfd14d64 00000100:00080000:1.0:1769074819.013754:0:27844:0:(import.c:65:import_set_state_nolock()) ffff89b5452ef000 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1769074819.013847:0:27844:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 4, last_transno is 17179874609 00000100:00100000:1.0:1769074819.013976:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+2:2405:x1855007616978304:12345-192.168.204.60@tcp:9:kworker.0 Request processed in 260us (868us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074819.013983:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14449 00010000:00010000:1.0:1769074819.014002:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074819.014021:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616977920 00000100:00100000:1.0:1769074819.014025:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616977920 00000100:00100000:1.0:1769074819.014070:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616978048 00000100:00100000:1.0:1769074819.014074:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616978048 00000100:00100000:1.0:1769074819.014098:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2398 00000100:00100000:1.0:1769074819.014105:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007616977920:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074819.014137:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007616977920:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (1207us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074819.014144:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2398 00000100:00100000:1.0:1769074819.014157:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2399 00000100:00100000:1.0:1769074819.014164:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616978048:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074819.014190:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616978048:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 27us (1114us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074819.014196:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2399 00000100:00080000:2.0:1769074819.015708:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074819.015718:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074819.015723:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074819.015726:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1769074819.487474:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074819.997357:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625212416:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074819.997470:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074819.997679:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625212416 00000100:00100000:3.0:1769074819.997692:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625212416 00000100:00100000:3.0:1769074819.997765:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2685 00000100:00100000:3.0:1769074819.997776:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625212416:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074819.997847:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074819.997912:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625212416:12345-0@lo:41:osp-pre-1-0.0 Request processed in 139us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074819.997921:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2685 00000100:00100000:1.0:1769074819.998266:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625212416:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074819.999497:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625212544:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074819.999585:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074819.999697:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625212544 00000100:00100000:0.0:1769074819.999707:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625212544 00000100:00100000:0.0:1769074819.999773:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2686 00000100:00100000:0.0:1769074819.999784:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625212544:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074819.999847:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445404672 avail 1299165184 00000100:00100000:0.0:1769074820.000009:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625212544:12345-0@lo:41:osp-pre-0-1.0 Request processed in 226us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074820.000017:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2686 00000100:00100000:1.0:1769074820.000104:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625212544:0@lo:41:osp-pre-0-1.0 00000001:02000400:2.0:1769074820.337540:0:37167:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-flr test_201 skipping ALWAYS excluded test 201 00010000:00010000:2.0:1769074820.509013:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074820.509313:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074820.509341:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625212672:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074820.509433:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074820.509529:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625212672 00000100:00100000:0.0:1769074820.509539:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625212672 00000100:00100000:0.0:1769074820.509606:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1204 00000100:00100000:0.0:1769074820.509617:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625212672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074820.509658:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 259392 pending 0 free 3917004800 avail 3685482496 00000100:00100000:0.0:1769074820.509709:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625212672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (276us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074820.509717:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1204 00000100:00100000:0.0:1769074820.509788:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa21c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625212672:0@lo:13:osp-pre-1-0.0 00000004:00080000:2.0:1769074820.510225:0:32395:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1808352416140ns, next at 1809352413082ns, rc = 0 00000100:00080000:2.0:1769074820.510254:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074820.510263:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074820.510274:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625212800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074820.510325:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074820.510368:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625212800 00000100:00100000:2.0:1769074820.510375:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625212800 00000100:00100000:2.0:1769074820.510415:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1205 00000100:00100000:2.0:1769074820.510424:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625212800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074820.510445:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3913945088 avail 3682422784 00000100:00100000:2.0:1769074820.510484:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625212800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074820.510491:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1205 00000100:00080000:2.0:1769074820.510523:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074820.510549:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625212800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074820.511560:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074820.511576:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625212928:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074820.511628:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074820.511675:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625212928 00000100:00100000:2.0:1769074820.511682:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625212928 00000100:00100000:2.0:1769074820.511723:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1206 00000100:00100000:2.0:1769074820.511731:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625212928:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074820.511753:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3913945088 avail 3682422784 00000100:00100000:2.0:1769074820.511848:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625212928:12345-0@lo:13:osp-pre-0-1.0 Request processed in 117us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074820.511865:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1206 00000100:00080000:2.0:1769074820.511888:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074820.511919:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625212928:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074821.021830:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074821.021873:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625213056:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074821.021973:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074821.022312:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625213056 00000100:00100000:3.0:1769074821.022324:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625213056 00000100:00100000:3.0:1769074821.022410:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1207 00000100:00100000:3.0:1769074821.022422:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625213056:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074821.022452:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 259392 pending 0 free 3917004800 avail 3685482496 00000100:00100000:3.0:1769074821.022524:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625213056:12345-0@lo:13:osp-pre-1-1.0 Request processed in 104us (550us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074821.022533:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1207 00000100:00100000:3.0:1769074821.022607:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625213056:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1769074821.023561:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1769074821.533276:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1769074821.853417:0:37215:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 202: lfs setstripe --add-component wide striping ========================================================== 04:40:20 (1769074820) 00000100:00100000:3.0:1769074822.287057:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.287152:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616978432 00000100:00100000:2.0:1769074822.287162:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616978432 00000100:00100000:2.0:1769074822.287256:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13425 00000100:00100000:2.0:1769074822.287264:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:64878:x1855007616978432:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074822.287280:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:3.0:1769074822.287290:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1769074822.287297:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54afb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4c7 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.287355:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287366:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287377:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287391:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287398:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287405:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287454:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287471:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287483:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287489:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287495:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.287499:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287506:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287519:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.287522:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.287527:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287533:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b02 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287547:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287553:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287557:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287563:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.287565:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287571:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287578:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b64effa800) 00010000:00010000:2.0:1769074822.287580:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.287582:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287587:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b09 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.287618:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54afb lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4c7 expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.287629:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.287631:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.287635:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54afb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4c7 expref: 8 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.287643:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074822.287699:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:64878:x1855007616978432:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 436us (653us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074822.287704:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13425 00000100:00100000:0.0:1769074822.300204:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.300357:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616978688 00000100:00100000:1.0:1769074822.300367:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616978688 00000100:00100000:1.0:1769074822.300430:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13426 00000100:00100000:1.0:1769074822.300439:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:64878:x1855007616978688:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.300461:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.300464:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074822.300499:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54ad8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4b9 expref: 7 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.300517:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.300522:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074822.300536:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b10 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4ce expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.300628:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.300641:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.300651:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.300664:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.300670:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.300676:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301537:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301549:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301557:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074822.301567:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00010000:00010000:1.0:1769074822.301588:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301595:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301601:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301607:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301613:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074822.301617:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301624:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301633:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.301635:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074822.301638:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301644:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b17 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000004:00010000:1.0:1769074822.301655:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000a26540cc reply state 0000000053a17704 transno 8589944048 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301669:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.301674:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:1.0:1769074822.301677:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1769074822.301679:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1769074822.301705:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b10 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.301721:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.301728:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b681697300) 00010000:00010000:1.0:1769074822.301730:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074822.301732:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521e00, rc 0) 00000100:00100000:1.0:1769074822.301801:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64878:x1855007616978688:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1363us (1606us total) trans 8589944048 rc 0/0 00000100:00100000:1.0:1769074822.301843:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13426 00000100:00100000:3.0:1769074822.316211:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.316385:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616978944 00000100:00100000:1.0:1769074822.316396:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616978944 00000100:00100000:1.0:1769074822.316467:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13427 00000100:00100000:1.0:1769074822.316486:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64878:x1855007616978944:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.316516:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.316542:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b33 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4d5 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.316666:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.316687:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.316696:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 4/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.316731:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.316744:0:10283:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 10 pid: 10283 timeout: 1910 lvb_type: 0 00000100:00100000:1.0:1769074822.316758:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625213184:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074822.316892:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074822.332545:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.332708:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625213184:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074822.332737:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074822.337100:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616979072 00000100:00100000:2.0:1769074822.337116:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616979072 00010000:00010000:2.0:1769074822.337167:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 11 pid: 10283 timeout: 1910 lvb_type: 0 00000100:00100000:2.0:1769074822.337217:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7189 00000100:00100000:2.0:1769074822.337227:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007616979072:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074822.337242:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074822.337249:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 11 pid: 10283 timeout: 1910 lvb_type: 0 00010000:00010000:2.0:1769074822.337264:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 11 pid: 10283 timeout: 1910 lvb_type: 0 00010000:00010000:2.0:1769074822.337285:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54b25 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ce expref: 11 pid: 10283 timeout: 1910 lvb_type: 0 00010000:00010000:2.0:1769074822.337303:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b681697300) 00010000:00010000:2.0:1769074822.337307:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.337316:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.337324:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.337357:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074822.337432:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007616979072:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 205us (4896us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074822.337440:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7189 00010000:00010000:1.0:1769074822.338456:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338469:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074822.338502:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00080000:1.0:1769074822.338674:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c3:0x0], index 0: 0 00010000:00010000:1.0:1769074822.338782:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338790:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338939:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338945:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074822.338949:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338959:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338970:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b681697300) 00010000:00010000:1.0:1769074822.338973:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074822.338976:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 2/0,0 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.338981:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b3a lrc: 0/0,0 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.339033:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b33 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4d5 expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.339043:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.339045:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074822.339048:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b33 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4d5 expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.339057:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641520000, rc 0) 00000100:00100000:1.0:1769074822.339309:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64878:x1855007616978944:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 22833us (23106us total) trans 8589944049 rc 301/301 00000100:00100000:1.0:1769074822.339318:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13427 00000100:00100000:0.0:1769074822.360643:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.362695:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616979328 00000100:00100000:3.0:1769074822.362712:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616979328 00000100:00100000:3.0:1769074822.362834:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4712 00000100:00100000:3.0:1769074822.362845:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64878:x1855007616979328:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074822.363144:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64878:x1855007616979328:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 300us (2516us total) trans 8589944050 rc 0/0 00000100:00100000:3.0:1769074822.363155:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4712 00000100:00100000:0.0:1769074822.367264:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.367354:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616979584 00000100:00100000:2.0:1769074822.367363:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616979584 00000100:00100000:2.0:1769074822.367414:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4713 00000100:00100000:2.0:1769074822.367420:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64878:x1855007616979584:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074822.367659:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:64878:x1855007616979584:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 237us (405us total) trans 8589944051 rc 0/0 00000100:00100000:2.0:1769074822.367667:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4713 00000100:00100000:1.0:1769074822.403143:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.403378:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616979712 00000100:00100000:3.0:1769074822.403389:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616979712 00000100:00100000:3.0:1769074822.403460:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13428 00000100:00100000:3.0:1769074822.403471:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:64885:x1855007616979712:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074822.403502:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074822.403522:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b48 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4dc expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.403587:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54b4f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.403601:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54b4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.403612:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54b4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074822.403658:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1769074822.403662:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074822.403681:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54b4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.403706:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b48 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4dc expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.403722:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54b4f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4dc expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.403730:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074822.403734:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074822.403738:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febac00, rc 0) 00000100:00100000:3.0:1769074822.403801:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64885:x1855007616979712:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 331us (667us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074822.403868:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13428 00000100:00100000:0.0:1769074822.422766:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.423026:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616979968 00000100:00100000:2.0:1769074822.423038:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616979968 00000100:00100000:2.0:1769074822.423107:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13429 00000100:00100000:2.0:1769074822.423118:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64885:x1855007616979968:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074822.423137:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074822.423157:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b56 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4e3 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.423230:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423244:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423255:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423265:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423271:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423277:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423359:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423367:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423373:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423380:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423385:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.423388:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423395:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423407:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.423411:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.423415:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423420:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b5d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423434:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423439:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423444:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423449:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.423452:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423457:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423464:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:2.0:1769074822.423466:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.423470:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423484:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b64 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.423512:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b56 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4e3 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.423521:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.423524:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074822.423527:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b56 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4e3 expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.423535:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074822.423600:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64885:x1855007616979968:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 483us (843us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074822.423607:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13429 00000100:00100000:0.0:1769074822.446677:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.449541:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616980096 00000100:00100000:3.0:1769074822.449564:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616980096 00000100:00100000:3.0:1769074822.449667:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13430 00000100:00100000:3.0:1769074822.449681:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64885:x1855007616980096:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074822.449786:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.449854:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.449871:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.449884:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.449891:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.449896:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450106:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450132:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450145:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074822.450194:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:2.0:1769074822.450199:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074822.450213:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450228:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450235:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450241:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450248:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450255:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.450259:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450266:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450279:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.450282:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.450287:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450294:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54b6b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450309:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450315:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450321:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450326:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.450329:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450511:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450521:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:2.0:1769074822.450524:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.450528:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450533:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54b72 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450542:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450547:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450556:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450560:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.450563:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450568:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450574:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074822.450576:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.450579:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.450584:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54b79 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074822.450745:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64885:x1855007616980096:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1063us (4072us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074822.450757:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13430 00000100:00100000:2.0:1769074822.530819:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.531061:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616980352 00000100:00100000:1.0:1769074822.531073:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616980352 00000100:00100000:1.0:1769074822.531138:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13431 00000100:00100000:1.0:1769074822.531149:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64893:x1855007616980352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.531170:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.531191:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54b80 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4ea expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.531265:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.531280:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.531291:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.531301:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.531308:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074822.531314:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532565:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532576:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532580:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532584:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532588:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.532591:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532595:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532604:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074822.532607:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.532612:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532618:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54b87 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532634:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532640:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532645:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532650:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641520000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.532652:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532657:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532663:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:0.0:1769074822.532666:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.532668:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532674:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54b8e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.532713:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54b80 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ea expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074822.532723:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074822.532726:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074822.532730:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54b80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ea expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074822.532739:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521400, rc 0) 00000100:00100000:0.0:1769074822.532850:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64893:x1855007616980352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1661us (2015us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074822.532859:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13431 00000100:00100000:0.0:1769074822.541209:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.541586:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616980480 00000100:00100000:2.0:1769074822.541600:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616980480 00000100:00100000:2.0:1769074822.541665:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13432 00000100:00100000:2.0:1769074822.541674:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64893:x1855007616980480:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074822.541750:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.541767:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.541779:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.541803:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.543094:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.543107:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543842:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543855:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543863:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074822.543911:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:0.0:1769074822.543916:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074822.543930:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543945:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543952:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543957:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543964:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543971:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.543974:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543981:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.543992:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074822.543996:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.544000:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544006:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54b95 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544020:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544026:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544031:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544036:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.544038:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544044:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544050:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:0.0:1769074822.544053:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.544056:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544061:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd54b9c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544069:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544074:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544080:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544321:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.544324:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544339:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544348:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:0.0:1769074822.544350:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.544354:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.544360:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54ba3 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:0.0:1769074822.544462:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64893:x1855007616980480:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 2784us (3260us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074822.544481:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13432 00010000:00010000:0.0:1769074822.557784:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074822.589354:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074822.589508:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616980608 00000100:00100000:0.0:1769074822.589518:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616980608 00000100:00100000:0.0:1769074822.589575:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13433 00000100:00100000:0.0:1769074822.589584:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:64894:x1855007616980608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074822.589603:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074822.589623:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54baa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4f1 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074822.589691:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589705:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589715:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589725:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589732:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589738:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589837:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589844:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589850:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074822.589891:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:0.0:1769074822.589895:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074822.589911:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589921:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589929:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589936:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589942:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589947:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589953:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.589957:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589963:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589973:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074822.589976:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.589979:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589985:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54bb1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.589998:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590004:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590010:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590014:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074822.590017:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590022:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590028:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:0.0:1769074822.590030:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074822.590032:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590037:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd54bb8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074822.590057:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd54baa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 14 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074822.590067:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074822.590073:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:0.0:1769074822.590076:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074822.590078:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165000, rc 0) 00000100:00100000:0.0:1769074822.590559:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:64894:x1855007616980608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 973us (1210us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074822.590568:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13433 00000100:00100000:2.0:1769074822.594238:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.594405:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616980736 00000100:00100000:3.0:1769074822.594419:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616980736 00000100:00100000:3.0:1769074822.594517:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14450 00000100:00100000:3.0:1769074822.594527:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616980736:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074822.594553:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074822.594580:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4f8 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074822.594610:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c3:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1769074822.594633:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074822.594635:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074822.594647:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 6 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074822.594668:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 6 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074822.594684:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb200, rc 0) 00000100:00100000:3.0:1769074822.594763:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007616980736:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 237us (542us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074822.594770:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14450 00000100:00080000:2.0:1769074822.595007:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074822.601339:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.601709:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616980992 00000100:00100000:3.0:1769074822.601724:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616980992 00000100:00100000:3.0:1769074822.601863:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13434 00000100:00100000:3.0:1769074822.601878:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:64894:x1855007616980992:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.601994:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.602032:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54bcd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c4ff expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.602158:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54bd4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.602172:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.602183:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074822.602196:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00010000:00010000:1.0:1769074822.602316:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54bcd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ff expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.602334:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54bd4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ff expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.602345:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:1.0:1769074822.602349:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074822.602363:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641520000, rc 0) 00000100:00100000:1.0:1769074822.602435:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:64894:x1855007616980992:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 561us (1112us total) trans 8589944052 rc 0/0 00000100:00100000:1.0:1769074822.602445:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13434 00000100:00100000:1.0:1769074822.609712:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.609961:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616981120 00000100:00100000:3.0:1769074822.609972:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616981120 00000100:00100000:3.0:1769074822.610036:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1043 00000100:00100000:3.0:1769074822.610047:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2407:x1855007616981120:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074822.610064:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074822.610087:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54bd4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4ff expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.610107:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074822.610113:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074822.610179:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2407:x1855007616981120:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 133us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074822.610186:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1043 00000100:00100000:2.0:1769074822.621664:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.621886:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616981376 00000100:00100000:3.0:1769074822.621896:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616981376 00000100:00100000:3.0:1769074822.621948:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4714 00000100:00100000:3.0:1769074822.621958:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:64894:x1855007616981376:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074822.622730:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:64894:x1855007616981376:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 769us (1072us total) trans 8589944053 rc 0/0 00000100:00100000:3.0:1769074822.622738:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4714 00000100:00100000:0.0:1769074822.629467:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.629639:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616981632 00000100:00100000:1.0:1769074822.629650:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616981632 00000100:00100000:1.0:1769074822.629708:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13435 00000100:00100000:1.0:1769074822.629718:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:64894:x1855007616981632:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.629742:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.629766:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54be2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c506 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.629927:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.629942:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 16 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.629951:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.629986:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 16 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.630000:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 16 pid: 10283 timeout: 1910 lvb_type: 0 00000100:00100000:1.0:1769074822.630012:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625213312:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074822.630087:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074822.633394:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.635950:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625213312:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074822.635980:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074822.636992:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616981760 00000100:00100000:3.0:1769074822.637002:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616981760 00010000:00010000:3.0:1769074822.637052:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 17 pid: 10283 timeout: 1910 lvb_type: 0 00000100:00100000:3.0:1769074822.637102:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7190 00000100:00100000:3.0:1769074822.637111:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616981760:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074822.637129:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074822.637136:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 17 pid: 10283 timeout: 1910 lvb_type: 0 00010000:00010000:3.0:1769074822.637147:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 17 pid: 10283 timeout: 1910 lvb_type: 0 00010000:00010000:3.0:1769074822.637168:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165000/0x7fe2323adfd54bbf lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f1 expref: 17 pid: 10283 timeout: 1910 lvb_type: 0 00010000:00010000:3.0:1769074822.637185:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074822.637189:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074822.637198:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074822.637205:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074822.637298:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074822.637548:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2409:x1855007616981760:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 435us (4160us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074822.637555:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7190 00010000:00010000:2.0:1769074822.641511:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.641529:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074822.641557:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00010000:00010000:2.0:1769074822.642766:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54be2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.642802:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.643240:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:2.0:1769074822.643243:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1769074822.643247:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641520000, rc 0) 00000100:00100000:2.0:1769074822.643329:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:64894:x1855007616981632:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 13610us (13870us total) trans 8589944054 rc 0/0 00000100:00100000:2.0:1769074822.643339:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13435 00000100:00100000:1.0:1769074822.646942:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.647109:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616981888 00000100:00100000:2.0:1769074822.647121:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616981888 00000100:00100000:2.0:1769074822.647181:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13436 00000100:00100000:2.0:1769074822.647191:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:64894:x1855007616981888:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074822.647215:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074822.647238:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54bf7 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c50d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.647331:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.647347:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.647369:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 4/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x13/0x8 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.647403:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.647416:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 17 pid: 6514 timeout: 1910 lvb_type: 0 00000100:00100000:2.0:1769074822.647430:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625213440:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074822.647520:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545ff8600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074822.653183:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625213440:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074822.653214:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x13/0x8 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:1.0:1769074822.661393:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074822.668849:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616982016 00000100:00100000:0.0:1769074822.668865:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616982016 00010000:00010000:0.0:1769074822.668918:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 18 pid: 6514 timeout: 1910 lvb_type: 0 00000100:00100000:0.0:1769074822.668964:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7191 00000100:00100000:0.0:1769074822.668974:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:2409:x1855007616982016:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074822.668991:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074822.668998:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 4/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 18 pid: 6514 timeout: 1910 lvb_type: 0 00010000:00010000:0.0:1769074822.669011:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 18 pid: 6514 timeout: 1910 lvb_type: 0 00010000:00010000:0.0:1769074822.669032:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54be9 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x200:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c506 expref: 18 pid: 6514 timeout: 1910 lvb_type: 0 00010000:00010000:0.0:1769074822.669049:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:0.0:1769074822.669054:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x13/0x8 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074822.669063:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074822.669073:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074822.669478:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074822.669542:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616982016:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 568us (8157us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074822.669551:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7191 00010000:00010000:2.0:1769074822.674026:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.674046:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074822.674103:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:2.0:1769074822.674108:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074822.674129:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.674734:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54bf7 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.674757:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.674765:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:2.0:1769074822.674768:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074822.674773:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:2.0:1769074822.675459:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:64894:x1855007616981888:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 28265us (28524us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074822.675482:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13436 00000100:00100000:1.0:1769074822.683456:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.683648:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616982272 00000100:00100000:2.0:1769074822.683658:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616982272 00000100:00100000:2.0:1769074822.683721:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13437 00000100:00100000:2.0:1769074822.683730:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:64894:x1855007616982272:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074822.683803:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.683859:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.683868:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.683899:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.683912:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 16 pid: 6514 timeout: 1910 lvb_type: 0 00000100:00100000:2.0:1769074822.683926:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fb97480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625213568:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074822.683999:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545ff8600 going to sleep for 11 seconds 00000100:00100000:3.0:1769074822.692437:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074822.692862:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616982400 00000100:00100000:0.0:1769074822.692873:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616982400 00010000:00010000:0.0:1769074822.692911:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 17 pid: 6514 timeout: 1910 lvb_type: 0 00000100:00100000:0.0:1769074822.692949:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7192 00000100:00100000:0.0:1769074822.692958:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616982400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074822.692974:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074822.692979:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 17 pid: 6514 timeout: 1910 lvb_type: 0 00010000:00010000:0.0:1769074822.692991:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 17 pid: 6514 timeout: 1910 lvb_type: 0 00010000:00010000:0.0:1769074822.693013:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:0.0:1769074822.693018:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074822.693027:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074822.693034:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074822.693052:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074822.693102:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007616982400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 145us (675us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074822.693109:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7192 00010000:00010000:1.0:1769074822.694696:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd54bfe lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c50d expref: 16 pid: 6514 timeout: 1910 lvb_type: 0 00000100:00100000:1.0:1769074822.694719:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fb97480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625213568:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074822.694740:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.694753:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.694758:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000001:00080000:1.0:1769074822.695173:0:6514:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944055 00000004:00010000:1.0:1769074822.695204:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000711a8532 reply state 000000008f78c25e transno 8589944056 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.695220:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.695228:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:1.0:1769074822.695231:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2a 00000100:00100000:1.0:1769074822.695235:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074822.695453:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:64894:x1855007616982272:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 11722us (12004us total) trans 8589944056 rc 0/0 00000100:00100000:1.0:1769074822.695462:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13437 00000020:00080000:1.0:1769074822.695467:0:6514:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1769074822.707590:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.707924:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616982656 00000100:00100000:3.0:1769074822.707935:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616982656 00000100:00100000:3.0:1769074822.707995:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4715 00000100:00100000:3.0:1769074822.708005:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:64894:x1855007616982656:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074822.708236:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64894:x1855007616982656:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 231us (653us total) trans 8589944057 rc 0/0 00000100:00100000:3.0:1769074822.708244:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4715 00000100:00100000:0.0:1769074822.770595:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.776015:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616982912 00000100:00100000:1.0:1769074822.776031:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616982912 00000100:00100000:1.0:1769074822.776111:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13438 00000100:00100000:1.0:1769074822.776122:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64896:x1855007616982912:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.776146:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.776168:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54c0c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c514 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.776241:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776261:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776272:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776701:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776714:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776722:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776797:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776803:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776927:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776934:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776940:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074822.776944:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776951:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776966:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.776969:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074822.776974:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776980:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c13 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.776995:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777000:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777004:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777009:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074822.777012:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777016:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777023:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:1.0:1769074822.777025:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074822.777028:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777033:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c1a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.777062:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54c0c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c514 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.777072:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.777074:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074822.777077:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54c0c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c514 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.777086:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521e00, rc 0) 00000100:00100000:1.0:1769074822.777216:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64896:x1855007616982912:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1094us (6633us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074822.777225:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13438 00000100:00100000:1.0:1769074822.792998:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.793303:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616983040 00000100:00100000:2.0:1769074822.793314:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616983040 00000100:00100000:2.0:1769074822.793385:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13439 00000100:00100000:2.0:1769074822.793396:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64896:x1855007616983040:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074822.793701:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793720:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793729:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793741:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793746:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793750:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793899:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793905:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793908:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074822.793941:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:2.0:1769074822.793944:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074822.793953:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793963:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793967:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793971:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793976:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793979:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.793983:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793988:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.793996:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.793998:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.794001:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794004:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c21 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794013:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794017:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794020:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794023:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.794025:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794029:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794032:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:2.0:1769074822.794034:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.794035:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794038:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c28 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794043:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794046:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794049:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794052:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.794053:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794056:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794060:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:2.0:1769074822.794061:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.794063:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074822.794069:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c2f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074822.794178:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64896:x1855007616983040:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 784us (1191us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074822.794184:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13439 00000100:00100000:2.0:1769074822.854238:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074822.854360:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616983296 00000100:00100000:1.0:1769074822.854370:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616983296 00000100:00100000:1.0:1769074822.854426:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13440 00000100:00100000:1.0:1769074822.854434:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64900:x1855007616983296:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074822.854454:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074822.854508:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54c36 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c51b expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.854583:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854597:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854620:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854633:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854640:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854648:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854720:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854728:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854734:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854740:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854746:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074822.854749:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854756:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854767:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.854770:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074822.854774:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854780:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54c3d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854794:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854800:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854852:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854856:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074822.854859:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854864:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854870:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:1.0:1769074822.854873:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074822.854875:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854880:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54c44 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074822.854908:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54c36 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c51b expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.854917:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074822.854919:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074822.854923:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54c36 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c51b expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074822.854931:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521e00, rc 0) 00000100:00100000:1.0:1769074822.854998:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64900:x1855007616983296:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 564us (769us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074822.855007:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13440 00000100:00100000:0.0:1769074822.868593:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.868793:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616983424 00000100:00100000:2.0:1769074822.868803:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616983424 00000100:00100000:2.0:1769074822.868962:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13441 00000100:00100000:2.0:1769074822.868972:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64900:x1855007616983424:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074822.869044:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869060:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869071:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869081:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869088:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869094:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869165:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869177:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869183:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074822.869223:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:2.0:1769074822.869227:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074822.869240:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869252:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869258:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869264:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869270:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869275:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.869279:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869285:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869295:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.869298:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.869303:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869309:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c4b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869323:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869328:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869334:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869338:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.869341:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869346:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869352:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:2.0:1769074822.869354:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.869357:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869362:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c52 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869372:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869377:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869388:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869393:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.869395:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869401:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869408:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:2.0:1769074822.869410:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.869413:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.869418:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c59 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074822.869638:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64900:x1855007616983424:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 665us (1053us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074822.869646:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13441 00000100:00100000:3.0:1769074822.931266:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074822.932689:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616983680 00000100:00100000:2.0:1769074822.932703:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616983680 00000100:00100000:2.0:1769074822.932777:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13442 00000100:00100000:2.0:1769074822.932787:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64902:x1855007616983680:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074822.932853:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074822.932876:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c60 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c522 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.932949:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.932963:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.932973:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.932984:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.932991:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.932996:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933077:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933084:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933091:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933097:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933103:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.933107:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933114:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933138:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.933142:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.933147:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.933153:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54c67 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934722:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934730:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934737:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934742:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074822.934745:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934752:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934766:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:2.0:1769074822.934769:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074822.934773:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.934778:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54c6e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074822.935490:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c60 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c522 expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.935504:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074822.935507:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074822.935510:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54c60 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c522 expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074822.935521:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074822.935607:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64902:x1855007616983680:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 2820us (4349us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074822.935617:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13442 00000100:00100000:2.0:1769074822.942537:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.942723:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616983808 00000100:00100000:3.0:1769074822.942736:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616983808 00000100:00100000:3.0:1769074822.942872:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13443 00000100:00100000:3.0:1769074822.942901:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64902:x1855007616983808:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074822.942984:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943000:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943012:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943022:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943029:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943036:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943111:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943119:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943125:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074822.943165:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00000004:00200000:3.0:1769074822.943169:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074822.943182:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943200:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943207:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943213:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943219:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943225:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074822.943228:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943235:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943246:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074822.943249:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074822.943254:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943259:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c75 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943272:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943277:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943283:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943288:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febba00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074822.943291:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943297:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943303:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:3.0:1769074822.943305:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074822.943308:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943315:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c7c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943322:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943328:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943333:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943338:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074822.943340:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943345:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943352:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074822.943355:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074822.943358:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.943363:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c83 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074822.943443:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64902:x1855007616983808:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 561us (920us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074822.943451:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13443 00000100:00100000:1.0:1769074822.998063:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074822.998158:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984064 00000100:00100000:3.0:1769074822.998170:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984064 00000100:00100000:3.0:1769074822.998235:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13444 00000100:00100000:3.0:1769074822.998245:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:64903:x1855007616984064:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074822.998266:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074822.998269:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074822.998296:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54b4f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4dc expref: 19 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.998314:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074822.998319:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074822.998332:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c8a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c529 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.998498:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998511:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998566:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998580:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998587:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998593:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998653:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54c9f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998661:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54c9f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998667:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54c9f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074822.998676:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 0 00010000:00010000:3.0:1769074822.998752:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998760:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998766:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998772:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998779:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febba00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074822.998783:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998790:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998799:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074822.998802:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074822.998860:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998867:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54c91 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998877:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998882:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998888:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998892:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074822.998895:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998901:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998908:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:3.0:1769074822.998910:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074822.998913:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998919:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54c98 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074822.998952:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54c8a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c529 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.998964:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54c9f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c529 expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074822.998971:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074822.998973:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074822.998976:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba600, rc 0) 00000100:00100000:3.0:1769074822.999042:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:64903:x1855007616984064:12345-192.168.204.60@tcp:101:dd.0 Request processed in 798us (988us total) trans 8589944058 rc 0/0 00000100:00100000:3.0:1769074822.999050:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13444 00000100:00100000:2.0:1769074823.003262:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074823.003611:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984192 00000100:00100000:1.0:1769074823.003622:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984192 00000100:00100000:1.0:1769074823.003686:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13445 00000100:00100000:1.0:1769074823.003696:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:64903:x1855007616984192:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074823.003720:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074823.003741:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54cad lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c530 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.003774:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54cb4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.003786:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.003798:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.003971:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54cad lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c530 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.003990:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521400/0x7fe2323adfd54cb4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c530 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.003998:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:1.0:1769074823.004001:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1769074823.004005:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521400, rc 0) 00000100:00100000:1.0:1769074823.004057:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:64903:x1855007616984192:12345-192.168.204.60@tcp:101:dd.0 Request processed in 363us (805us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074823.004065:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13445 00000100:00100000:2.0:1769074823.015394:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.017727:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984448 00000100:00100000:3.0:1769074823.017741:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984448 00000100:00100000:3.0:1769074823.017882:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13446 00000100:00100000:3.0:1769074823.017891:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:64903:x1855007616984448:12345-192.168.204.60@tcp:36:dd.0 00010000:00010000:3.0:1769074823.017946:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074823.017974:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54c9f lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c529 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.017991:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.017996:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074823.018152:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,1 mode: --/PW res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.018166:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.018178:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000001:00080000:3.0:1769074823.018223:0:10283:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944059 00000004:00010000:3.0:1769074823.018245:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000cd846d28 reply state 00000000b3459bab transno 8589944060 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.018257:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.018263:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.018265:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074823.018268:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074823.018997:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fb97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:64903:x1855007616984448:12345-192.168.204.60@tcp:36:dd.0 Request processed in 1104us (3609us total) trans 8589944060 rc 0/0 00000100:00100000:3.0:1769074823.019006:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13446 00000020:00080000:3.0:1769074823.019012:0:10283:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1769074823.029978:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.030060:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984576 00000100:00100000:3.0:1769074823.030070:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984576 00000100:00100000:3.0:1769074823.030164:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13447 00000100:00100000:3.0:1769074823.030174:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:64903:x1855007616984576:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074823.030199:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074823.030221:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54cc2 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c537 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.030278:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54cc2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c537 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.030302:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54cc2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c537 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.030321:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.030324:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074823.030328:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba800, rc 0) 00000100:00100000:3.0:1769074823.030399:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:64903:x1855007616984576:12345-192.168.204.60@tcp:101:dd.0 Request processed in 226us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074823.030406:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13447 00000100:00100000:2.0:1769074823.037773:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074823.037897:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984704 00000100:00100000:0.0:1769074823.037907:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984704 00000100:00100000:0.0:1769074823.037968:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14451 00000100:00100000:0.0:1769074823.037978:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:64903:x1855007616984704:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:0.0:1769074823.037997:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074823.038014:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54cc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c53e expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074823.038046:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074823.038056:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074823.038058:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074823.038062:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54cc9 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000080000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c53e expref: 7 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074823.038096:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074823.038109:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 1910 lvb_type: 1 00000100:00100000:0.0:1769074823.038127:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625213696:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074823.038216:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5499cb180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074823.044922:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625213696:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074823.044958:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54cc9 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000080020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c53e expref: 7 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074823.044977:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164a00, rc 0) 00000100:00100000:0.0:1769074823.045041:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:64903:x1855007616984704:12345-192.168.204.60@tcp:101:dd.0 Request processed in 7062us (7276us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.045050:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14451 00000100:00100000:1.0:1769074823.046751:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.046847:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984832 00000100:00100000:3.0:1769074823.046857:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984832 00010000:00010000:3.0:1769074823.046894:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 1910 lvb_type: 1 00000100:00100000:3.0:1769074823.046937:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7193 00000100:00100000:3.0:1769074823.046947:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007616984832:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074823.046963:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074823.046968:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 1910 lvb_type: 1 00010000:00010000:3.0:1769074823.046981:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 1910 lvb_type: 1 00010000:00010000:3.0:1769074823.047003:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54bc6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c4f8 expref: 7 pid: 28519 timeout: 1910 lvb_type: 1 00010000:00010000:3.0:1769074823.047018:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074823.047020:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074823.047027:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54cc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080080020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c53e expref: 6 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.047037:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54cc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080080020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c53e expref: 6 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.047073:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b682164a00/0x7fe2323adfd54cc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000080020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c53e expref: 6 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074823.047087:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625213824:192.168.204.60@tcp:105: 00000100:00100000:3.0:1769074823.047176:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b681626480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074823.071289:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625214080:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071362:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074823.071402:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625214592:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071426:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074823.071442:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625215104:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071473:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074823.071502:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074823.071509:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625214208:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071532:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074823.071547:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625214720:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071566:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074823.071583:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625215232:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071607:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074823.071643:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214080 00000100:00100000:2.0:1769074823.071650:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214080 00000100:00100000:2.0:1769074823.071692:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215104 00000100:00100000:2.0:1769074823.071695:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215104 00000100:00100000:2.0:1769074823.071715:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2687 00000100:00100000:2.0:1769074823.071723:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625214080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071773:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625214080:12345-0@lo:400:kworker.0 Request processed in 50us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.071780:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2687 00000100:00100000:2.0:1769074823.071799:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2688 00000100:00100000:2.0:1769074823.071838:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625215104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.071868:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625215104:12345-0@lo:400:kworker.0 Request processed in 64us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.071874:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2688 00000100:00100000:2.0:1769074823.071892:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214592 00000100:00100000:2.0:1769074823.071895:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214592 00000100:00100000:2.0:1769074823.071927:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214208 00000100:00100000:2.0:1769074823.071931:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214208 00000100:00100000:2.0:1769074823.071949:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214720 00000100:00100000:2.0:1769074823.071952:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214720 00000100:00100000:2.0:1769074823.071968:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2400 00000100:00100000:2.0:1769074823.071975:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625214592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072003:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625214592:12345-0@lo:400:kworker.0 Request processed in 29us (576us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.072009:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2400 00000100:00100000:2.0:1769074823.072021:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2401 00000100:00100000:2.0:1769074823.072027:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625214208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072053:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625214208:12345-0@lo:400:kworker.0 Request processed in 27us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.072059:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2401 00000100:00100000:2.0:1769074823.072350:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2402 00000100:00100000:2.0:1769074823.072357:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625214720:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072388:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625214720:12345-0@lo:400:kworker.0 Request processed in 32us (821us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.072393:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2402 00000100:00080000:2.0:1769074823.072432:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074823.072436:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074823.072476:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625214208:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072488:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625214720:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072498:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625214080:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072506:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625214592:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.072514:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625215104:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.072722:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625213952:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.072761:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.072801:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625214464:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.072856:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.072878:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625214976:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.072897:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.072914:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625215488:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.072933:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.072951:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1769074823.072957:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625214336:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.072978:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.072995:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625214848:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073017:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.073034:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625215360:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073054:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074823.073080:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215232 00000100:00100000:0.0:1769074823.073086:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215232 00000100:00100000:0.0:1769074823.073121:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214464 00000100:00100000:0.0:1769074823.073124:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214464 00000100:00100000:0.0:1769074823.073142:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214336 00000100:00100000:0.0:1769074823.073144:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214336 00000100:00100000:0.0:1769074823.073162:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215360 00000100:00100000:0.0:1769074823.073164:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215360 00000100:00100000:0.0:1769074823.073182:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1507 00000100:00100000:0.0:1769074823.073188:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625215232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073226:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625215232:12345-0@lo:400:kworker.0 Request processed in 39us (1618us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073233:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1507 00000100:00100000:0.0:1769074823.073245:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1508 00000100:00100000:0.0:1769074823.073250:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625214464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073281:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625214464:12345-0@lo:400:kworker.0 Request processed in 30us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073287:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1508 00000100:00100000:0.0:1769074823.073297:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1509 00000100:00100000:0.0:1769074823.073303:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625214336:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073326:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625214336:12345-0@lo:400:kworker.0 Request processed in 24us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073332:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1509 00000100:00100000:0.0:1769074823.073341:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1510 00000100:00100000:0.0:1769074823.073347:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625215360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073596:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625215360:12345-0@lo:400:kworker.0 Request processed in 248us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073602:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1510 00000100:00100000:0.0:1769074823.073643:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214976 00000100:00100000:0.0:1769074823.073647:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214976 00000100:00100000:0.0:1769074823.073667:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215488 00000100:00100000:0.0:1769074823.073670:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215488 00000100:00100000:0.0:1769074823.073679:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625214848 00000100:00100000:0.0:1769074823.073681:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625214848 00000100:00100000:0.0:1769074823.073693:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2403 00000100:00100000:0.0:1769074823.073698:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625214976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073732:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625214976:12345-0@lo:400:kworker.0 Request processed in 34us (834us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073737:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2403 00000100:00100000:0.0:1769074823.073748:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2404 00000100:00100000:0.0:1769074823.073753:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625215488:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073778:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625215488:12345-0@lo:400:kworker.0 Request processed in 26us (844us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073784:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2404 00000100:00100000:0.0:1769074823.073794:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2405 00000100:00100000:0.0:1769074823.073800:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625214848:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073853:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625214848:12345-0@lo:400:kworker.0 Request processed in 54us (835us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074823.073859:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2405 00000100:00080000:0.0:1769074823.073887:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074823.073891:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074823.073894:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074823.073896:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074823.073923:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625214336:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073933:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625214848:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073943:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625215360:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073952:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625214464:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073960:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625214976:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.073969:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625215488:0@lo:400:kworker.0 00000100:00100000:0.0:1769074823.074008:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625215232:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.074207:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625213952 00000100:00100000:2.0:1769074823.074213:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625213952 00000100:00100000:2.0:1769074823.074252:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1769074823.074258:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545175450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625213952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.074300:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545175450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625213952:12345-0@lo:400:kworker.0 Request processed in 42us (1539us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.074306:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1769074823.074343:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625213952:0@lo:400:kworker.0 00000100:00100000:2.0:1769074823.078425:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.078596:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625213824:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074823.078622:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074823.078683:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616984832:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 31736us (31939us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074823.078693:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7193 00000100:00100000:0.0:1769074823.079149:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616984960 00000100:00100000:0.0:1769074823.079156:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616984960 00002000:00010000:0.0:1769074823.079192:0:8404:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/707 (0->18446744073709551615) 00002000:00100000:0.0:1769074823.079197:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1de680 with x1855007616984960 ext(0->18446744073709551615) 00002000:00010000:0.0:1769074823.079209:0:8404:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1de680. 00000100:00100000:0.0:1769074823.079240:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8742 00000100:00100000:0.0:1769074823.079250:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616984960:12345-192.168.204.60@tcp:10:dd.0 00002000:00010000:0.0:1769074823.079440:0:8404:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c3:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1769074823.079444:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c3:0x0] updating lvb mtime from disk: 0 -> 1769074822 00002000:00010000:0.0:1769074823.079446:0:8404:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c3:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074823.079448:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c3:0x0] updating lvb ctime from disk: 0 -> 1769074822 00002000:00010000:0.0:1769074823.079450:0:8404:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c3:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074823.080036:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007616984960:12345-192.168.204.60@tcp:10:dd.0 Request processed in 785us (1620us total) trans 21474841283 rc 0/0 00000100:00100000:0.0:1769074823.080047:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8742 00002000:00010000:0.0:1769074823.080051:0:8404:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/707 (0->18446744073709551615) 00002000:00100000:0.0:1769074823.080054:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1de680 with x1855007616984960 ext(0->18446744073709551615) 00002000:00010000:0.0:1769074823.080059:0:8404:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1de680. 00000001:00080000:3.0:1769074823.086782:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841283 is committed 00000001:00100000:3.0:1769074823.086798:0:24714:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000020:00000080:3.0:1769074823.086802:0:24714:0:(genops.c:904:class_export_put()) final put ffff89b649256800/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000001:00080000:3.0:1769074823.088359:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841282 is committed 00000020:00000080:2.0:1769074823.088669:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b649256800/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 for lustre-OST0000 00000020:00000080:2.0:1769074823.088739:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b649251800 for lustre-OST0000 00000100:00100000:2.0:1769074823.141550:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.141683:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616985216 00000100:00100000:3.0:1769074823.141695:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616985216 00000100:00100000:3.0:1769074823.141764:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13448 00000100:00100000:3.0:1769074823.141774:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:64903:x1855007616985216:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074823.141795:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074823.141798:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074823.141875:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54cc2 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c537 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.141899:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.141904:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074823.141917:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54cd0 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c545 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.141966:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54cd7 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.141982:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54cd7 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.141991:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54cd7 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x200:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00080000:3.0:1769074823.142124:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2c6:0x0], index 1: 0 00000004:00080000:3.0:1769074823.142131:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c4:0x0], index 0: 0 00010000:00010000:3.0:1769074823.142237:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54cd7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x200:0x0].0x0 bits 0xa/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c545 expref: 26 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.142246:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.142249:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074823.142258:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54cd0 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x200:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c545 expref: 26 pid: 10283 timeout: 0 lvb_type: 3 00010000:00010000:3.0:1769074823.142270:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba600/0x7fe2323adfd54cd7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x200:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c545 expref: 25 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.142288:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.142291:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074823.142293:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba600, rc 0) 00000100:00100000:3.0:1769074823.142358:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:64903:x1855007616985216:12345-192.168.204.60@tcp:101:dd.0 Request processed in 586us (818us total) trans 8589944061 rc 0/0 00000100:00100000:3.0:1769074823.142366:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13448 00000001:00080000:2.0:1769074823.145069:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944061 is committed 00000004:00080000:2.0:1769074823.145104:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1960 committed 00000001:00100000:2.0:1769074823.145115:0:32377:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000001:00080000:2.0:1769074823.145120:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944047 is committed 00000020:00000080:2.0:1769074823.145123:0:32377:0:(genops.c:904:class_export_put()) final put ffff89b543683000/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00010000:00010000:1.0:1769074823.145204:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145216:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145224:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145230:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641520e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074823.145235:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145242:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145258:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:1.0:1769074823.145262:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074823.145266:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145272:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54b1e lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145309:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145316:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145326:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145330:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074823.145333:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145338:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145344:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:1.0:1769074823.145346:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2a 00010000:00010000:1.0:1769074823.145350:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145366:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54c05 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2a/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.145377:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145383:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145388:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145392:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074823.145395:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145400:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145409:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:1.0:1769074823.145411:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074823.145413:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.145417:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd54cbb lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000020:00000080:3.0:1769074823.145646:0:35568:0:(genops.c:863:class_export_destroy()) destroying export ffff89b543683000/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 for lustre-MDT0000 00000020:00000080:3.0:1769074823.145703:0:35568:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b643aae800 for lustre-MDT0000 00000100:00100000:1.0:1769074823.148625:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074823.148657:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625215616:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1769074823.148738:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074823.148955:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215616 00000100:00100000:3.0:1769074823.148964:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215616 00000100:00100000:3.0:1769074823.149021:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14452 00000100:00100000:3.0:1769074823.149031:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625215616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1769074823.149073:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:706 count 1 00002000:00010000:3.0:1769074823.149109:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c2:0x0] initial LVB size: 2097152, mtime: 0x6971f06e, atime: 0x6971f06e, ctime: 0x6971f073, blocks: 0x1000 00010000:00010000:3.0:1769074823.149117:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.149133:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074823.149135:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074823.149141:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.149155:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.149164:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.149171:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074823.149200:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1769074823.152143:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074823.152159:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.152172:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febba00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074823.152176:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.152187:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.152208:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.152216:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febba00/0x7fe2323adfd54cde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.152243:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074823.152246:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074823.152249:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074823.156213:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625215616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7171us (7461us total) trans 21474841284 rc 0/0 00000100:00100000:3.0:1769074823.156226:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14452 00000100:00100000:1.0:1769074823.156589:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841282 gen 2 00000004:00080000:1.0:1769074823.156597:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c68700, transno 21474841282 00000100:00100000:1.0:1769074823.156618:0:3650:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b547ef4000 x1855007625215616/t21474841284(21474841284) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1769074839 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1769074823.156628:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef4000/2, rc 0, transno 21474841284 00000100:00100000:1.0:1769074823.156633:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625215616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1769074823.159304:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074823.159389:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616985344 00000100:00100000:1.0:1769074823.159399:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616985344 00002000:00100000:1.0:1769074823.159447:0:28434:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x2c3:0x0] (0->1048575) req@ffff89b54d1ddc00 x1855007616985344/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:359/0 lens 488/0 e 0 to 0 dl 1769074834 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074823.159458:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1ddc00 with x1855007616985344 ext(0->1048575) 00002000:00010000:1.0:1769074823.159473:0:28434:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1ddc00 00000100:00100000:1.0:1769074823.159508:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8743 00000100:00100000:1.0:1769074823.159517:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2406:x1855007616985344:12345-192.168.204.60@tcp:4:dd.0 00000100:00100000:1.0:1769074823.310103:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007616985344:12345-192.168.204.60@tcp:4:dd.0 Request processed in 150574us (150796us total) trans 21474841285 rc 0/0 00000100:00100000:1.0:1769074823.310120:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8743 00002000:00100000:1.0:1769074823.310133:0:28434:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x2c3:0x0] (0->1048575) req@ffff89b54d1ddc00 x1855007616985344/t21474841285(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:359/0 lens 488/448 e 0 to 0 dl 1769074834 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769074823.310143:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1ddc00 with x1855007616985344 ext(0->1048575) 00002000:00010000:1.0:1769074823.310149:0:28434:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b54d1ddc00 00000100:00100000:2.0:1769074823.317536:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.317652:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616985472 00000100:00100000:3.0:1769074823.317662:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616985472 00000100:00100000:3.0:1769074823.317710:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14453 00000100:00100000:3.0:1769074823.317718:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0+-99:2405:x1855007616985472:12345-192.168.204.60@tcp:8:dd.0 00010000:00080000:3.0:1769074823.317752:0:27843:0:(ldlm_lib.c:1428:target_handle_connect()) lustre-OST0001: connection from 249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp t0 exp 0000000000000000 cur 1811 last 0 00000020:00000080:3.0:1769074823.317783:0:27843:0:(genops.c:1402:class_connect()) connect: client 249ac4ab-c06c-4166-99d5-c4c7e1cc2791, cookie 0x7fe2323adfd54ce5 00002000:00100000:3.0:1769074823.317796:0:27843:0:(ofd_obd.c:124:ofd_parse_connect_data()) lustre-OST0001: cli 249ac4ab-c06c-4166-99d5-c4c7e1cc2791/ffff89b66e78f000 ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 2113204 ocd_grant: 484638720 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1769074823.317847:0:27843:0:(ofd_obd.c:222:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00000020:01000000:3.0:1769074823.317919:0:27843:0:(lprocfs_status_server.c:658:lprocfs_exp_setup()) using hash ffff89b66e700200 00002000:00080000:3.0:1769074823.317930:0:27843:0:(ofd_obd.c:365:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1769074823.317959:0:27843:0:(import.c:65:import_set_state_nolock()) ffff89b66e78e000 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1769074823.318030:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2405:x1855007616985472:12345-192.168.204.60@tcp:8:dd.0 Request processed in 310us (503us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074823.318038:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14453 00000100:00100000:1.0:1769074823.321345:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074823.321663:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616985600 00000100:00100000:2.0:1769074823.321674:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616985600 00000100:00100000:2.0:1769074823.321762:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14454 00000100:00100000:2.0:1769074823.321773:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:64903:x1855007616985600:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074823.321793:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074823.321855:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd54cec lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c54c expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074823.321929:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c6:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1769074823.321951:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074823.321954:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074823.321962:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd54cec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c54c expref: 7 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.321981:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89600/0x7fe2323adfd54cec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c54c expref: 7 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.321993:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89600, rc 0) 00000100:00100000:2.0:1769074823.322319:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:64903:x1855007616985600:12345-192.168.204.60@tcp:101:dd.0 Request processed in 545us (983us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.322327:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14454 00000100:00080000:3.0:1769074823.323126:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00100000:1.0:1769074823.326408:0:26036:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 249ac4ab-c06c-4166-99d5-c4c7e1cc2791 00000001:00100000:1.0:1769074823.326420:0:26036:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 249ac4ab-c06c-4166-99d5-c4c7e1cc2791 00000020:00000080:1.0:1769074823.326424:0:26036:0:(genops.c:904:class_export_put()) final put ffff89b5452ef800/249ac4ab-c06c-4166-99d5-c4c7e1cc2791 00000001:00100000:1.0:1769074823.326453:0:26036:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000020:00000080:1.0:1769074823.326456:0:26036:0:(genops.c:904:class_export_put()) final put ffff89b66e785000/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000001:00080000:1.0:1769074823.326476:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874609 is committed 00000020:00000080:3.0:1769074823.326548:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b5452ef800/249ac4ab-c06c-4166-99d5-c4c7e1cc2791 for lustre-OST0001 00000020:00000080:3.0:1769074823.334163:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b66e785000/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 for lustre-OST0001 00000020:00000080:2.0:1769074823.334513:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b5452ef000 for lustre-OST0001 00000020:00000080:2.0:1769074823.334520:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b66e781000 for lustre-OST0001 00000100:00100000:2.0:1769074823.355183:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074823.355273:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616985856 00000100:00100000:0.0:1769074823.355282:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616985856 00000100:00100000:0.0:1769074823.355335:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4716 00000100:00100000:0.0:1769074823.355343:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:64903:x1855007616985856:12345-192.168.204.60@tcp:35:dd.0 00000100:00100000:0.0:1769074823.355685:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64903:x1855007616985856:12345-192.168.204.60@tcp:35:dd.0 Request processed in 342us (509us total) trans 8589944062 rc 0/0 00000100:00100000:0.0:1769074823.355693:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4716 00000100:00100000:2.0:1769074823.427857:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074823.428087:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616985984 00000100:00100000:1.0:1769074823.428097:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616985984 00000100:00100000:1.0:1769074823.428158:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13449 00000100:00100000:1.0:1769074823.428168:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:64905:x1855007616985984:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074823.428191:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074823.428212:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54cf3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c553 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.428276:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54cfa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.428290:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54cfa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.428303:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54cfa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:1.0:1769074823.428345:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:1.0:1769074823.428349:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074823.428368:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54cfa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074823.428391:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54cf3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c553 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.428410:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54cfa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c553 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.428419:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074823.428422:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1769074823.428426:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641520000, rc 0) 00000100:00100000:1.0:1769074823.429127:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64905:x1855007616985984:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 955us (1329us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074823.429136:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13449 00000100:00100000:1.0:1769074823.443892:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074823.444034:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616986240 00000100:00100000:2.0:1769074823.444046:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616986240 00000100:00100000:2.0:1769074823.444120:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13450 00000100:00100000:2.0:1769074823.444131:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64905:x1855007616986240:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074823.444153:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074823.444178:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d01 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c55a expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.444264:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444291:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444302:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444318:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444325:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444331:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444406:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444413:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444418:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444425:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444430:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.444435:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444441:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444453:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074823.444466:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.444471:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444478:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d08 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444493:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444499:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444504:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444509:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.444512:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444518:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444528:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b64effaa00) 00010000:00010000:2.0:1769074823.444531:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.444533:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444539:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d0f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.444569:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d01 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c55a expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.444578:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074823.444581:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074823.444584:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d01 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c55a expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.444595:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074823.444664:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64905:x1855007616986240:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 535us (781us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074823.444671:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13450 00000100:00100000:2.0:1769074823.447788:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.448407:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616986368 00000100:00100000:3.0:1769074823.448419:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616986368 00000100:00100000:3.0:1769074823.448494:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13451 00000100:00100000:3.0:1769074823.448507:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64905:x1855007616986368:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074823.448600:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448622:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448634:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448651:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448658:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448665:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448745:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448753:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.448759:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074823.448799:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 2097152 00000004:00200000:3.0:1769074823.449080:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074823.449097:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449112:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449123:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449129:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449136:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449142:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074823.449146:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449153:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449166:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074823.449169:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074823.449173:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449179:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449192:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449198:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449204:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449208:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074823.449211:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449216:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449222:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681690500) 00010000:00010000:3.0:1769074823.449225:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074823.449227:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449233:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d1d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449246:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449251:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449257:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449261:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074823.449264:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449269:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449913:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.449917:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074823.449921:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.449930:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d24 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074823.450029:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64905:x1855007616986368:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1521us (2250us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074823.450038:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13451 00000100:00100000:0.0:1769074823.489685:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074823.489785:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616986624 00000100:00100000:2.0:1769074823.489795:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616986624 00000100:00100000:2.0:1769074823.489903:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13452 00000100:00100000:2.0:1769074823.489914:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64909:x1855007616986624:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074823.489935:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074823.489955:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d2b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c561 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.490034:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490050:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490061:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490075:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490082:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490087:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490156:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490163:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490168:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490174:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490179:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.490183:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490194:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490205:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074823.490208:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.490213:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490219:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d32 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490231:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490236:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490240:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490245:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.490247:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490252:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490258:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b64effaa00) 00010000:00010000:2.0:1769074823.490259:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.490262:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490266:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d39 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.490513:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d2b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c561 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.490525:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074823.490527:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074823.490531:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d2b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c561 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074823.490540:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074823.490611:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64909:x1855007616986624:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 699us (935us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074823.490619:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13452 00000100:00100000:1.0:1769074823.500135:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074823.500658:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616986752 00000100:00100000:3.0:1769074823.500669:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616986752 00000100:00100000:3.0:1769074823.500729:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13453 00000100:00100000:3.0:1769074823.500740:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64909:x1855007616986752:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074823.500849:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500865:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500877:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500891:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500898:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500903:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500973:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500982:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.500989:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074823.501031:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 2097152 00000004:00200000:3.0:1769074823.501036:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074823.501049:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501063:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501069:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501075:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501081:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501086:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074823.501090:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501097:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501109:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074823.501112:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074823.501117:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501122:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d40 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501137:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501142:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501148:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501152:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074823.501155:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501160:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501166:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681690500) 00010000:00010000:3.0:1769074823.501168:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074823.501171:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501175:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d47 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501188:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501193:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501198:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501205:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074823.501208:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501212:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501218:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:3.0:1769074823.501220:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074823.501222:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074823.501227:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d4e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:1.0:1769074823.505116:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64909:x1855007616986752:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 4364us (4981us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074823.505132:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13453 00000100:00100000:0.0:1769074823.532452:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074823.532532:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987008 00000100:00100000:1.0:1769074823.532542:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987008 00000100:00100000:1.0:1769074823.532600:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13454 00000100:00100000:1.0:1769074823.532610:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64911:x1855007616987008:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074823.532631:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074823.532655:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54d55 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c568 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.532731:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532746:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532757:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532772:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532779:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532785:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532904:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532912:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532918:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532924:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532930:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641521e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074823.532934:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532942:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532954:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074823.532957:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074823.532962:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532968:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521e00/0x7fe2323adfd54d5c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532983:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532989:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.532995:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.533000:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641520e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074823.533002:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.533008:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.533014:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b681697c00) 00010000:00010000:1.0:1769074823.533017:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074823.533019:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.533025:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54d63 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074823.533060:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54d55 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c568 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.533070:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074823.533072:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074823.533076:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641521000/0x7fe2323adfd54d55 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c568 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074823.533084:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521000, rc 0) 00000100:00100000:1.0:1769074823.533133:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64911:x1855007616987008:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 525us (700us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074823.533140:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13454 00000100:00100000:0.0:1769074823.541509:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074823.541613:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987136 00000100:00100000:2.0:1769074823.541623:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987136 00000100:00100000:2.0:1769074823.541685:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13455 00000100:00100000:2.0:1769074823.541695:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64911:x1855007616987136:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074823.541771:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541787:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541799:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541850:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541858:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541865:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541929:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541938:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.541945:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074823.541985:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x200:0x0]: returning size 2097152 00000004:00200000:2.0:1769074823.541989:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074823.542002:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542015:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542022:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542029:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542035:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542040:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.542044:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542052:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542064:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074823.542067:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.542072:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542078:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54d6a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542092:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542098:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542104:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542109:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.542112:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542117:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542123:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xfb2b0516 (ffff89b64effaa00) 00010000:00010000:2.0:1769074823.542126:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.542129:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542135:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54d71 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xfb2b0516 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542149:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542155:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542161:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542166:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074823.542169:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542174:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542181:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x200:0x0].0x0 (ffff89b67a317d00) 00010000:00010000:2.0:1769074823.542183:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074823.542185:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074823.542191:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54d78 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x200:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074823.542254:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:64911:x1855007616987136:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 559us (754us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074823.542262:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13455 00010000:00010000:3.0:1769074823.581712:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:0.0:1769074824.100889:0:7431:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-MDT0001: committing for initial connect of 2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000001:00080000:0.0:1769074824.100908:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968349 is committed 00000020:00000080:0.0:1769074824.100914:0:7431:0:(genops.c:904:class_export_put()) final put ffff89b643aab000/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 00000020:00000080:2.0:1769074824.107494:0:31965:0:(genops.c:863:class_export_destroy()) destroying export ffff89b643aab000/2274e3cf-6162-4197-bb1e-d97abc9b4bb9 for lustre-MDT0001 00000020:00000080:2.0:1769074824.115928:0:31965:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff89b54361a000 for lustre-MDT0001 00000100:00100000:1.0:1769074824.149014:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074824.149051:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074824.149082:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074824.149106:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987648 00000100:00100000:1.0:1769074824.149115:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987648 00000100:00100000:1.0:1769074824.149192:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1511 00000100:00100000:1.0:1769074824.149202:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007616987648:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074824.149283:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007616987648:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 82us (200us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074824.149290:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1511 00000100:00100000:1.0:1769074824.149318:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987392 00000100:00100000:1.0:1769074824.149322:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987392 00000100:00100000:1.0:1769074824.149357:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2406 00000100:00100000:1.0:1769074824.149364:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007616987392:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074824.149393:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007616987392:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 30us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074824.149398:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2406 00000100:00100000:0.0:1769074824.149557:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987264 00000100:00100000:0.0:1769074824.149562:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987264 00000100:00100000:0.0:1769074824.149609:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 363 00000100:00100000:0.0:1769074824.149616:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545176850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007616987264:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074824.149641:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545176850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007616987264:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (638us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074824.149647:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 363 00000100:00080000:2.0:1769074824.150377:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074824.150383:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074824.164847:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074824.164911:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074824.164988:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987520 00000100:00100000:2.0:1769074824.164999:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987520 00000100:00100000:2.0:1769074824.165076:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2407 00000100:00100000:2.0:1769074824.165088:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616987520:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074824.165166:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2409:x1855007616987520:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 80us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074824.165174:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2407 00000100:00080000:0.0:1769074824.165402:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074824.166486:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987776 00000100:00100000:3.0:1769074824.166498:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987776 00000100:00100000:3.0:1769074824.166552:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1512 00000100:00100000:3.0:1769074824.166562:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007616987776:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074824.167037:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007616987776:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 474us (2123us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074824.167045:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1512 00010000:00010000:0.0:1769074824.619211:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074825.117959:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074825.117983:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625215744:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074825.118058:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074825.118193:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074825.118203:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625215872:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074825.118232:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074825.118262:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215744 00000100:00100000:0.0:1769074825.118268:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215744 00000100:00100000:0.0:1769074825.118320:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625215872 00000100:00100000:0.0:1769074825.118324:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625215872 00000100:00100000:0.0:1769074825.118343:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2689 00000100:00100000:0.0:1769074825.118351:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625215744:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074825.118383:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446150144 avail 1299910656 00000100:00100000:0.0:1769074825.118426:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625215744:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074825.118444:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2689 00000100:00100000:0.0:1769074825.118461:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2690 00000100:00100000:0.0:1769074825.118467:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625215872:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074825.118482:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445404672 avail 1299165184 00000100:00100000:0.0:1769074825.118512:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625215872:12345-0@lo:41:osp-pre-0-1.0 Request processed in 45us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074825.118518:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2690 00000100:00100000:0.0:1769074825.118559:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625215872:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074825.118599:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625215744:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074825.629079:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074825.629103:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625216000:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074825.629179:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074825.629411:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216000 00000100:00100000:3.0:1769074825.629431:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216000 00000100:00100000:3.0:1769074825.629496:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1208 00000100:00100000:3.0:1769074825.629507:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625216000:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769074825.629536:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 8697152 pending 0 free 3917004800 avail 3685482496 00000100:00100000:3.0:1769074825.629584:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625216000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074825.629591:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1208 00000100:00100000:3.0:1769074825.629630:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874609 gen 2 00000004:00080000:3.0:1769074825.629636:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67fa20700, transno 17179874609 00000100:00100000:3.0:1769074825.629651:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625216000:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1769074825.629722:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074825.630416:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074825.630483:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625216256:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074825.630524:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074825.630554:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074825.630559:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625216128:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074825.630581:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074825.630612:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216256 00000100:00100000:3.0:1769074825.630617:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216256 00000100:00100000:3.0:1769074825.630650:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216128 00000100:00100000:3.0:1769074825.630653:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216128 00000100:00100000:3.0:1769074825.630671:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1209 00000100:00100000:3.0:1769074825.630678:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625216256:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074825.630698:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 485971264 pending 1073152 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074825.630733:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625216256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 55us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074825.630739:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1209 00000100:00100000:3.0:1769074825.630753:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1210 00000100:00100000:3.0:1769074825.630759:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625216128:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074825.630769:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 485971264 pending 1073152 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074825.630790:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625216128:12345-0@lo:13:osp-pre-0-1.0 Request processed in 32us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074825.630796:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1210 00000100:00100000:3.0:1769074825.630848:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625216128:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074825.630862:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625216256:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1769074825.632934:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074826.141329:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074826.141354:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625216384:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074826.141601:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074826.141829:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216384 00000100:00100000:0.0:1769074826.141838:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216384 00000100:00100000:0.0:1769074826.141905:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1211 00000100:00100000:0.0:1769074826.141915:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625216384:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074826.141936:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 8697152 pending 0 free 3917004800 avail 3685482496 00000100:00100000:0.0:1769074826.141983:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625216384:12345-0@lo:13:osp-pre-1-1.0 Request processed in 69us (543us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074826.141990:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1211 00000100:00080000:3.0:1769074826.142087:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074826.142139:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625216384:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1769074826.660363:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074827.677239:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074828.192623:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625216896:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.192694:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.192747:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625217408:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.192772:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.192789:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625217920:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.192841:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.192866:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1769074828.192872:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625216512:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.192898:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.192927:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625217024:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.192947:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.192964:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625217536:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.192983:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.192998:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625218048:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193015:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193042:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625216640:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193064:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193082:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625217152:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193101:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193115:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625217664:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193133:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193151:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074828.193156:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625216768:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193177:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193192:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625217280:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193212:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193221:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625217792:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193239:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074828.193271:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216896 00000100:00100000:2.0:1769074828.193278:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216896 00000100:00100000:2.0:1769074828.193329:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217920 00000100:00100000:2.0:1769074828.193333:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217920 00000100:00100000:2.0:1769074828.193351:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217024 00000100:00100000:2.0:1769074828.193354:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217024 00000100:00100000:2.0:1769074828.193391:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217792 00000100:00100000:2.0:1769074828.193394:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217792 00000100:00100000:2.0:1769074828.193416:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1513 00000100:00100000:2.0:1769074828.193424:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625216896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193477:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625216896:12345-0@lo:400:kworker.0 Request processed in 53us (782us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193484:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1513 00000100:00100000:2.0:1769074828.193500:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1514 00000100:00100000:2.0:1769074828.193507:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625217920:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193536:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625217920:12345-0@lo:400:kworker.0 Request processed in 29us (694us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193541:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1514 00000100:00100000:2.0:1769074828.193551:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1515 00000100:00100000:2.0:1769074828.193557:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625217024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193587:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625217024:12345-0@lo:400:kworker.0 Request processed in 30us (639us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193592:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1515 00000100:00100000:2.0:1769074828.193602:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1516 00000100:00100000:2.0:1769074828.193608:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625217792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193638:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625217792:12345-0@lo:400:kworker.0 Request processed in 31us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193643:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1516 00000100:00100000:2.0:1769074828.193661:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217408 00000100:00100000:2.0:1769074828.193664:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217408 00000100:00100000:2.0:1769074828.193685:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217536 00000100:00100000:2.0:1769074828.193688:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217536 00000100:00100000:2.0:1769074828.193699:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625218048 00000100:00100000:2.0:1769074828.193702:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625218048 00000100:00100000:2.0:1769074828.193710:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217152 00000100:00100000:2.0:1769074828.193712:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217152 00000100:00100000:2.0:1769074828.193722:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216768 00000100:00100000:2.0:1769074828.193724:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216768 00000100:00100000:2.0:1769074828.193733:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217280 00000100:00100000:2.0:1769074828.193735:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217280 00000100:00100000:2.0:1769074828.193748:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2408 00000100:00100000:2.0:1769074828.193753:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625217408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193783:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625217408:12345-0@lo:400:kworker.0 Request processed in 30us (1010us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193788:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2408 00000100:00100000:2.0:1769074828.193799:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2409 00000100:00100000:2.0:1769074828.193825:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625217536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193853:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625217536:12345-0@lo:400:kworker.0 Request processed in 49us (869us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193858:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2409 00000100:00100000:2.0:1769074828.193869:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2410 00000100:00100000:2.0:1769074828.193874:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625218048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193898:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625218048:12345-0@lo:400:kworker.0 Request processed in 24us (882us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193904:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2410 00000100:00100000:2.0:1769074828.193914:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2411 00000100:00100000:2.0:1769074828.193920:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625217152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193944:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625217152:12345-0@lo:400:kworker.0 Request processed in 25us (842us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193949:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2411 00000100:00100000:2.0:1769074828.193958:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2412 00000100:00100000:2.0:1769074828.193964:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625216768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.193987:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625216768:12345-0@lo:400:kworker.0 Request processed in 24us (810us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.193992:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2412 00000100:00100000:2.0:1769074828.194002:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2413 00000100:00100000:2.0:1769074828.194008:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625217280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194032:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625217280:12345-0@lo:400:kworker.0 Request processed in 25us (819us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.194037:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2413 00000100:00100000:2.0:1769074828.194062:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216512 00000100:00100000:2.0:1769074828.194065:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216512 00000100:00100000:2.0:1769074828.194092:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1769074828.194097:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649911850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625216512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194123:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649911850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625216512:12345-0@lo:400:kworker.0 Request processed in 26us (1225us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.194129:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1769074828.194172:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625216640 00000100:00100000:2.0:1769074828.194176:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625216640 00000100:00100000:2.0:1769074828.194196:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625217664 00000100:00100000:2.0:1769074828.194199:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625217664 00000100:00100000:2.0:1769074828.194213:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2691 00000100:00100000:2.0:1769074828.194219:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625216640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194249:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625216640:12345-0@lo:400:kworker.0 Request processed in 29us (1183us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.194254:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2691 00000100:00100000:2.0:1769074828.194266:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2692 00000100:00100000:2.0:1769074828.194272:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625217664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194298:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625217664:12345-0@lo:400:kworker.0 Request processed in 27us (1164us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074828.194304:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2692 00000100:00080000:2.0:1769074828.194327:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074828.194332:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074828.194334:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074828.194337:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074828.194396:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625216768:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194408:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625217280:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194418:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625217792:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194426:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625216640:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194434:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625217152:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194442:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625217664:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194481:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625216512:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194491:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625217024:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194501:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625217536:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194510:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625218048:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194520:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625216896:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194530:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625217408:0@lo:400:kworker.0 00000100:00100000:2.0:1769074828.194540:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625217920:0@lo:400:kworker.0 00010000:00010000:2.0:1769074828.700974:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1769074829.217260:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944062 is committed 00000001:00080000:1.0:1769074829.229274:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841285 is committed 00000001:00080000:1.0:1769074829.229292:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841284 is committed 00000100:00100000:0.0:1769074829.255043:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074829.255089:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074829.255127:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074829.255157:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074829.255197:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074829.255229:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616988160 00000100:00100000:0.0:1769074829.255239:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616988160 00000100:00100000:0.0:1769074829.255297:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616988032 00000100:00100000:0.0:1769074829.255301:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616988032 00000100:00100000:0.0:1769074829.255325:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2414 00000100:00100000:0.0:1769074829.255334:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616988160:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074829.255419:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007616988160:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 85us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074829.255426:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2414 00000100:00100000:0.0:1769074829.255584:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2415 00000100:00100000:0.0:1769074829.255592:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007616988032:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074829.255621:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007616988032:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 30us (463us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074829.255627:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2415 00000100:00100000:0.0:1769074829.255649:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616988416 00000100:00100000:0.0:1769074829.255653:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616988416 00000100:00100000:0.0:1769074829.255680:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616988288 00000100:00100000:0.0:1769074829.255683:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616988288 00000100:00100000:0.0:1769074829.255794:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1517 00000100:00100000:0.0:1769074829.255804:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616988416:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074829.255865:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616988416:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 64us (738us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074829.255871:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1517 00000100:00100000:0.0:1769074829.255882:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1518 00000100:00100000:0.0:1769074829.255887:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616988288:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074829.255906:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616988288:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 19us (709us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074829.255911:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1518 00000100:00080000:0.0:1769074829.255925:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074829.255929:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074829.255931:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074829.255933:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074829.262060:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616987904 00000100:00100000:2.0:1769074829.262074:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616987904 00000100:00100000:2.0:1769074829.262144:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 365 00000100:00100000:2.0:1769074829.262154:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649913850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007616987904:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074829.262239:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649913850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007616987904:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 86us (7205us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074829.262247:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 365 00010000:00010000:3.0:1769074829.731788:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1769074830.077125:0:37411:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 203: mirror file preserve mirror ID === 04:40:28 (1769074828) 00000100:00100000:0.0:1769074830.172041:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.172123:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616988544 00000100:00100000:3.0:1769074830.172133:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616988544 00000100:00100000:3.0:1769074830.172411:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13456 00000100:00100000:3.0:1769074830.172422:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:65489:x1855007616988544:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.172456:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.172478:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d7f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c56f expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.172547:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172562:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172574:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172589:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172595:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172601:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172643:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172651:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172657:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172662:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172668:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.172671:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172677:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172939:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.172943:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.172949:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172955:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd54d86 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172968:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172974:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172979:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172984:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.172987:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.172992:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.173001:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690a00) 00010000:00010000:3.0:1769074830.173002:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.173005:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.173010:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54d8d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.173039:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d7f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c56f expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.173048:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.173051:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.173054:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54d7f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c56f expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.173062:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba800, rc 0) 00000100:00100000:3.0:1769074830.173125:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:65489:x1855007616988544:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 706us (1096us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074830.173134:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13456 00000100:00100000:2.0:1769074830.199753:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.199959:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616988800 00000100:00100000:3.0:1769074830.199970:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616988800 00000100:00100000:3.0:1769074830.200026:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13457 00000100:00100000:3.0:1769074830.200036:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:65489:x1855007616988800:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.200056:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.200059:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074830.200084:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520000/0x7fe2323adfd54cfa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c553 expref: 9 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.200103:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.200107:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074830.200121:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54d94 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c576 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.200352:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.200367:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.200376:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.200389:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.200395:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.200401:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.208898:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.208927:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.208938:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.208949:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:3.0:1769074830.208976:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.208982:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.208988:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.208995:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.209000:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.209004:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.209010:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.209021:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.209023:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.209027:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.209032:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54d9b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000004:00010000:3.0:1769074830.209047:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000006cfdc19d reply state 00000000ed31ffd8 transno 8589944063 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.209056:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.209061:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:3.0:1769074830.209063:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074830.209066:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1769074830.209102:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54d94 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.209116:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.209122:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690a00) 00010000:00010000:3.0:1769074830.209125:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074830.209127:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba800, rc 0) 00000100:00100000:3.0:1769074830.209589:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:65489:x1855007616988800:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9551us (9843us total) trans 8589944063 rc 0/0 00000100:00100000:3.0:1769074830.209599:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13457 00000100:00100000:3.0:1769074830.213128:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.213206:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616989056 00000100:00100000:0.0:1769074830.213216:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616989056 00000100:00100000:0.0:1769074830.213278:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13458 00000100:00100000:0.0:1769074830.213288:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:65489:x1855007616989056:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074830.213312:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074830.213334:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd54db7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c57d expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.213467:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.213483:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.213490:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 4/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.213521:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.213534:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 12 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:0.0:1769074830.213548:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625218176:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074830.213624:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b654182540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074830.217432:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.217476:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616989184 00000100:00100000:3.0:1769074830.217486:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616989184 00010000:00010000:3.0:1769074830.217524:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 13 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:3.0:1769074830.217559:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7194 00000100:00100000:3.0:1769074830.217570:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2408:x1855007616989184:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074830.217585:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074830.217591:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 13 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.217603:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 13 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.217623:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690a00) 00010000:00010000:3.0:1769074830.217629:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.217637:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.217644:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.217661:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074830.217714:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2408:x1855007616989184:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 144us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.217721:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7194 00010000:00010000:0.0:1769074830.219257:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd54da9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c576 expref: 12 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:0.0:1769074830.219275:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625218176:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074830.219294:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219302:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219307:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:0.0:1769074830.219324:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00080000:0.0:1769074830.219514:0:6513:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2c7:0x0], index 1: 0 00000004:00080000:0.0:1769074830.219522:0:6513:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c5:0x0], index 0: 0 00010000:00010000:0.0:1769074830.219616:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219624:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219630:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219634:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.219638:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219644:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219657:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690a00) 00010000:00010000:0.0:1769074830.219660:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074830.219664:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 2/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219668:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54dbe lrc: 0/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.219716:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd54db7 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c57d expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.219725:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074830.219727:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074830.219731:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd54db7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c57d expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.219743:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165600, rc 0) 00000100:00100000:0.0:1769074830.222405:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:65489:x1855007616989056:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6519us (6688us total) trans 8589944064 rc 301/301 00000100:00100000:0.0:1769074830.222423:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13458 00000100:00100000:3.0:1769074830.225355:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.225604:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616989440 00000100:00100000:0.0:1769074830.225615:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616989440 00000100:00100000:0.0:1769074830.225678:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4717 00000100:00100000:0.0:1769074830.225698:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:65489:x1855007616989440:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074830.226016:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:65489:x1855007616989440:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 329us (684us total) trans 8589944065 rc 0/0 00000100:00100000:0.0:1769074830.226024:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4717 00000100:00100000:3.0:1769074830.228569:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.228650:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616989696 00000100:00100000:1.0:1769074830.228661:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616989696 00000100:00100000:1.0:1769074830.228725:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4718 00000100:00100000:1.0:1769074830.228735:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:65489:x1855007616989696:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074830.228993:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:65489:x1855007616989696:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 258us (432us total) trans 8589944066 rc 0/0 00000100:00100000:1.0:1769074830.229001:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4718 00000100:00100000:2.0:1769074830.240039:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625218304:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074830.240109:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074830.240158:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074830.240175:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625218432:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074830.240197:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074830.240238:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625218304 00000100:00100000:2.0:1769074830.240246:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625218304 00000100:00100000:2.0:1769074830.240285:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625218432 00000100:00100000:2.0:1769074830.240289:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625218432 00000100:00100000:2.0:1769074830.240309:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2693 00000100:00100000:2.0:1769074830.240317:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625218304:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074830.240361:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446150144 avail 1299910656 00000100:00100000:2.0:1769074830.240407:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625218304:12345-0@lo:41:osp-pre-1-0.0 Request processed in 90us (297us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.240413:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2693 00000100:00100000:2.0:1769074830.240431:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2694 00000100:00100000:2.0:1769074830.240450:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625218432:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074830.240468:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445404672 avail 1299165184 00000100:00100000:2.0:1769074830.240496:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625218432:12345-0@lo:41:osp-pre-0-1.0 Request processed in 59us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.240502:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2694 00000100:00100000:2.0:1769074830.240547:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625218432:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074830.240559:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625218304:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074830.258026:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.261598:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616989824 00000100:00100000:2.0:1769074830.261614:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616989824 00000100:00100000:2.0:1769074830.261684:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13459 00000100:00100000:2.0:1769074830.261994:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:65491:x1855007616989824:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.262020:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.262041:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54dcc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c584 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.262104:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54dd3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.262119:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54dd3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.262132:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54dd3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.262318:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074830.262322:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074830.262352:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54dd3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.262377:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54dcc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c584 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.262393:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54dd3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c584 expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.262401:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.262403:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074830.262407:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074830.262616:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:65491:x1855007616989824:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 626us (4598us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.262624:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13459 00000100:00100000:3.0:1769074830.272396:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.274049:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616990080 00000100:00100000:0.0:1769074830.274062:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616990080 00000100:00100000:0.0:1769074830.274118:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13460 00000100:00100000:0.0:1769074830.274128:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:65491:x1855007616990080:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074830.274148:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074830.274178:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd54dda lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c58b expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.274253:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274268:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274279:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274290:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274296:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274302:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274422:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274431:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.274781:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:0.0:1769074830.274795:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:0.0:1769074830.275012:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275020:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275025:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275031:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275036:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.275040:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275050:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275061:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074830.275063:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.275069:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275075:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd54de1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275085:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275090:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275094:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275098:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.275100:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275104:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275110:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:0.0:1769074830.275112:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.275114:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275118:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54de8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.275228:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd54dda lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 17 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.275246:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.275253:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:0.0:1769074830.275255:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074830.275257:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165a00, rc 0) 00000100:00100000:0.0:1769074830.275380:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:65491:x1855007616990080:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1253us (2992us total) trans 8589944067 rc 0/0 00000100:00100000:0.0:1769074830.275389:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13460 00000100:00100000:2.0:1769074830.283482:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.284105:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616990208 00000100:00100000:3.0:1769074830.284116:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616990208 00000100:00100000:3.0:1769074830.284183:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14455 00000100:00100000:3.0:1769074830.284193:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2406:x1855007616990208:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.284210:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.284229:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c592 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074830.284253:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c7:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1769074830.284271:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.284273:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.284281:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.284298:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.284310:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb200, rc 0) 00000100:00100000:3.0:1769074830.284379:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007616990208:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 188us (906us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.284386:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14455 00000100:00100000:3.0:1769074830.286579:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.286865:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616990464 00000100:00100000:1.0:1769074830.286876:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616990464 00000100:00100000:1.0:1769074830.286924:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4719 00000100:00100000:1.0:1769074830.286934:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:65491:x1855007616990464:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074830.287151:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:65491:x1855007616990464:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 217us (576us total) trans 8589944068 rc 0/0 00000100:00100000:1.0:1769074830.287174:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4719 00000100:00100000:2.0:1769074830.289469:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.289537:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616990720 00000100:00100000:3.0:1769074830.289546:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616990720 00000100:00100000:3.0:1769074830.289647:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13461 00000100:00100000:3.0:1769074830.289659:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:65491:x1855007616990720:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.289682:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.289702:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54e04 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c599 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.289823:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54e0b lrc: 3/1,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.289839:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54e0b lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.289850:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54e0b lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.289861:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:3.0:1769074830.289962:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54e04 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c599 expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.289977:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54e0b lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c599 expref: 19 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.289986:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:3.0:1769074830.289989:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1769074830.289992:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba200, rc 0) 00000100:00100000:3.0:1769074830.290056:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65491:x1855007616990720:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 399us (596us total) trans 8589944069 rc 0/0 00000100:00100000:3.0:1769074830.294573:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13461 00000100:00100000:1.0:1769074830.295119:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.295196:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616990976 00000100:00100000:2.0:1769074830.295206:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616990976 00000100:00100000:2.0:1769074830.295252:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13462 00000100:00100000:2.0:1769074830.295263:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65491:x1855007616990976:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.295287:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.295306:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e19 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5a0 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.295425:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.295440:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.295449:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 4/0,1 mode: --/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.295486:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.295500:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 20 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:2.0:1769074830.295513:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625218560:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074830.295598:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e240 going to sleep for 11 seconds 00000100:00100000:1.0:1769074830.319528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.319564:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616991104 00000100:00100000:1.0:1769074830.319574:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616991104 00010000:00010000:1.0:1769074830.319610:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 21 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:1.0:1769074830.319645:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7195 00000100:00100000:1.0:1769074830.319655:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616991104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074830.319671:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074830.319679:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 5/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 21 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:1.0:1769074830.319692:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 21 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:1.0:1769074830.319727:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:1.0:1769074830.319732:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,1 mode: --/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074830.319741:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074830.319749:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074830.319769:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074830.319833:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007616991104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 179us (314us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074830.319842:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7195 00010000:00010000:3.0:1769074830.321448:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54def lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c58b expref: 20 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:3.0:1769074830.321469:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625218560:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074830.321493:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.321502:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.321512:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.321531:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:3.0:1769074830.321657:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e19 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a0 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.321674:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 3/0,0 mode: CW/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a0 expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.321682:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:3.0:1769074830.321685:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1769074830.321689:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:3.0:1769074830.321743:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:65491:x1855007616990976:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 26483us (26626us total) trans 8589944070 rc 0/0 00000100:00100000:3.0:1769074830.321752:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13462 00000100:00100000:1.0:1769074830.327043:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.327095:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616991232 00000100:00100000:2.0:1769074830.327105:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616991232 00000100:00100000:2.0:1769074830.327208:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4720 00000100:00100000:2.0:1769074830.327230:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:65491:x1855007616991232:12345-192.168.204.60@tcp:33:lfs.0 00000100:00100000:2.0:1769074830.327408:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:65491:x1855007616991232:12345-192.168.204.60@tcp:33:lfs.0 Request processed in 179us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.327414:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4720 00000100:00100000:3.0:1769074830.333499:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.333576:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616991488 00000100:00100000:2.0:1769074830.333586:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616991488 00000100:00100000:2.0:1769074830.333649:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4721 00000100:00100000:2.0:1769074830.333658:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:65491:x1855007616991488:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074830.333920:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:65491:x1855007616991488:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 262us (429us total) trans 8589944071 rc 0/0 00000100:00100000:2.0:1769074830.333928:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4721 00000100:00100000:1.0:1769074830.343729:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.344213:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616991744 00000100:00100000:3.0:1769074830.344223:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616991744 00000100:00100000:3.0:1769074830.344285:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13463 00000100:00100000:3.0:1769074830.344295:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:65491:x1855007616991744:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.344322:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.344325:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 1 00010000:00010000:3.0:1769074830.344381:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e20 lrc: 0/0,0 mode: --/CW res: [0x200000405:0x202:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a0 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.344406:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54e0b lrc: 0/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c599 expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.344416:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:3.0:1769074830.344427:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074830.344453:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e2e lrc: 2/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5a7 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.344553:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd54e35 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.344566:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd54e35 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.344575:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd54e35 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.344587:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:3.0:1769074830.344684:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e2e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a7 expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.344711:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd54e35 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a7 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.344719:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690c00) 00010000:00010000:3.0:1769074830.344721:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074830.344724:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb600, rc 0) 00000100:00100000:3.0:1769074830.344784:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:65491:x1855007616991744:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 490us (1064us total) trans 8589944072 rc 0/0 00000100:00100000:3.0:1769074830.344791:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13463 00000100:00100000:3.0:1769074830.348776:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.349229:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616992000 00000100:00100000:0.0:1769074830.349240:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616992000 00000100:00100000:0.0:1769074830.349289:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4722 00000100:00100000:0.0:1769074830.349298:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:65491:x1855007616992000:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:0.0:1769074830.349378:0:15947:0:(mdt_open.c:2313:mdt_close_handle_layouts()) ### [0x200000405:0x202:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd54e35 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a7 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.349415:0:15947:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349426:0:15947:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349434:0:15947:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349594:0:15947:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349602:0:15947:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349608:0:15947:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349614:0:15947:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.349618:0:15947:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349624:0:15947:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349634:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690c00) 00010000:00010000:0.0:1769074830.349637:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1769074830.349642:0:15947:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349648:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd54e43 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.349659:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690c00) 00010000:00010000:0.0:1769074830.349661:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074830.349665:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd54e35 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5a7 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074830.349892:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:65491:x1855007616992000:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 594us (1127us total) trans 8589944073 rc -22/-22 00000100:00100000:0.0:1769074830.349900:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4722 00000100:00100000:1.0:1769074830.354635:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.354720:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616992128 00000100:00100000:3.0:1769074830.354730:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616992128 00000100:00100000:3.0:1769074830.354795:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13464 00000100:00100000:3.0:1769074830.354839:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:65491:x1855007616992128:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.354862:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.354884:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 2/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5ae expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.354940:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.354955:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.354973:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690d00) 00010000:00010000:3.0:1769074830.354976:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074830.354980:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febac00, rc 0) 00000100:00100000:3.0:1769074830.355039:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:65491:x1855007616992128:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 235us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.355046:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13464 00000100:00100000:3.0:1769074830.359620:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.359976:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616992384 00000100:00100000:2.0:1769074830.359988:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616992384 00000100:00100000:2.0:1769074830.360047:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13465 00000100:00100000:2.0:1769074830.360058:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:65491:x1855007616992384:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.360078:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.360082:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074830.360111:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54dd3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c584 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.360129:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.360134:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074830.360151:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e51 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5b5 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.361073:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x203:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.361087:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.361097:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.361108:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x203:0x0]: returning size 0 00010000:00010000:2.0:1769074830.361152:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e51 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.361411:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.361419:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x203:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074830.361421:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074830.361423:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89c00, rc 0) 00000100:00100000:2.0:1769074830.361681:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:65491:x1855007616992384:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1623us (2069us total) trans 8589944074 rc 0/0 00000100:00100000:2.0:1769074830.361699:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13465 00000100:00100000:1.0:1769074830.365151:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.365249:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616992640 00000100:00100000:2.0:1769074830.365264:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616992640 00000100:00100000:2.0:1769074830.365322:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13466 00000100:00100000:2.0:1769074830.365332:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:65491:x1855007616992640:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.365366:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.365388:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54e66 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5bc expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.365510:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e6d lrc: 3/0,1 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.365525:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e6d lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.365536:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e6d lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.365547:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:2.0:1769074830.365648:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54e66 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5bc expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.365666:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e6d lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5bc expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.365674:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690d00) 00010000:00010000:2.0:1769074830.365677:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074830.365680:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074830.365762:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:65491:x1855007616992640:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 432us (621us total) trans 8589944075 rc 0/0 00000100:00100000:2.0:1769074830.365770:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13466 00000100:00100000:1.0:1769074830.369291:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.369914:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616992896 00000100:00100000:2.0:1769074830.369926:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616992896 00000100:00100000:2.0:1769074830.369986:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4723 00000100:00100000:2.0:1769074830.369996:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:65491:x1855007616992896:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:2.0:1769074830.370076:0:6515:0:(mdt_open.c:2313:mdt_close_handle_layouts()) ### [0x200000405:0x202:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e6d lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5bc expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.370117:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.370130:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.370138:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 4/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.370185:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.370197:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 25 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:2.0:1769074830.370211:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625218688:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074830.370284:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074830.373554:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.373663:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625218688:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074830.373702:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:3.0:1769074830.373843:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616993024 00000100:00100000:3.0:1769074830.373851:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616993024 00010000:00010000:3.0:1769074830.373886:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 26 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:3.0:1769074830.373918:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7196 00000100:00100000:3.0:1769074830.373927:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efff050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2408:x1855007616993024:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074830.373943:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074830.373950:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 26 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.373962:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 26 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.373983:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e4a lrc: 0/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ae expref: 26 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.373998:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690d00) 00010000:00010000:3.0:1769074830.374003:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.374012:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.374018:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.374041:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074830.374100:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efff050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2408:x1855007616993024:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 173us (555us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.374107:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7196 00010000:00010000:2.0:1769074830.374268:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.374276:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.374296:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x203:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.374312:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.374320:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.374356:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.374368:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 24 pid: 6514 timeout: 1918 lvb_type: 0 00000100:00100000:2.0:1769074830.374381:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625218816:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074830.374452:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074830.378027:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625218816:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074830.378057:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:0.0:1769074830.379374:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.379976:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616993152 00000100:00100000:1.0:1769074830.379987:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616993152 00010000:00010000:1.0:1769074830.380026:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 25 pid: 6514 timeout: 1918 lvb_type: 0 00000100:00100000:1.0:1769074830.380145:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7197 00000100:00100000:1.0:1769074830.380155:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007616993152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074830.380170:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074830.380177:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 25 pid: 6514 timeout: 1918 lvb_type: 0 00010000:00010000:1.0:1769074830.380190:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 25 pid: 6514 timeout: 1918 lvb_type: 0 00010000:00010000:1.0:1769074830.380213:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd54e58 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5b5 expref: 25 pid: 6514 timeout: 1918 lvb_type: 0 00010000:00010000:1.0:1769074830.380228:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x203:0x0].0x0 (ffff89b64effa700) 00010000:00010000:1.0:1769074830.380233:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.380247:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.380253:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.380283:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074830.380338:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007616993152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 185us (971us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074830.380355:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7197 00010000:00010000:2.0:1769074830.380404:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.380412:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:2.0:1769074830.380729:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625218944:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074830.380779:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074830.380852:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8540 going to sleep for 11 seconds 00000100:00100000:2.0:1769074830.380870:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625218944 00000100:00100000:2.0:1769074830.380875:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625218944 00000100:00100000:2.0:1769074830.380911:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2695 00000100:00100000:2.0:1769074830.380918:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625218944:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1769074830.380994:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625218944:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 76us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.381000:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2695 00000100:00080000:2.0:1769074830.381020:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074830.381060:0:6515:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841284 gen 2 00000004:00080000:2.0:1769074830.381066:0:6515:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef4000, transno 21474841284 00000100:00100000:2.0:1769074830.381079:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625218944:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1769074830.381124:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944076 00000001:00080000:2.0:1769074830.381164:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944077 00010000:00010000:2.0:1769074830.381183:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381190:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381196:0:6515:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381202:0:6515:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.381206:0:6515:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381213:0:6515:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381224:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x203:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074830.381227:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1769074830.381230:0:6515:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381237:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54e82 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x203:0x0].0x0 bits 0x28/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381252:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381258:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381264:0:6515:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381269:0:6515:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.381271:0:6515:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381277:0:6515:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381284:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690d00) 00010000:00010000:2.0:1769074830.381286:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1769074830.381289:0:6515:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 2/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381294:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e7b lrc: 0/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.381305:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690d00) 00010000:00010000:2.0:1769074830.381307:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074830.381312:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e6d lrc: 0/0,0 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5bc expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074830.381389:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:65491:x1855007616992896:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 11395us (12108us total) trans 8589944078 rc 0/0 00000100:00100000:2.0:1769074830.381397:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4723 00000020:00080000:2.0:1769074830.381401:0:6515:0:(tgt_main.c:782:tgt_ses_key_exit()) total 3 transactions per RPC 00000100:00100000:0.0:1769074830.387720:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.387972:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616993280 00000100:00100000:3.0:1769074830.387984:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616993280 00000100:00100000:3.0:1769074830.388057:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13467 00000100:00100000:3.0:1769074830.388068:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:65491:x1855007616993280:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.388093:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.388119:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5c3 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.388193:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 26 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.388208:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 26 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.388225:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:3.0:1769074830.388228:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074830.388233:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febac00, rc 0) 00000100:00100000:3.0:1769074830.388303:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:65491:x1855007616993280:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 237us (594us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.388311:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13467 00000100:00100000:0.0:1769074830.392043:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.392152:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616993536 00000100:00100000:1.0:1769074830.392173:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616993536 00000100:00100000:1.0:1769074830.392224:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4724 00000100:00100000:1.0:1769074830.392233:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:65491:x1855007616993536:12345-192.168.204.60@tcp:35:lfs.0 00000004:00080000:1.0:1769074830.392548:0:6515:0:(mdd_object.c:3993:mdd_close()) Object [0x200000405:0x203:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1769074830.392609:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2467, 64 off211048 00000004:00080000:1.0:1769074830.392620:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1818234812398ns, next at 1819234810192ns, rc = 0 00000001:00080000:1.0:1769074830.392627:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944079 00010000:00010000:1.0:1769074830.392666:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x203:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.392681:0:6515:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.392928:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.392939:0:6515:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.392946:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.392953:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.392977:0:6515:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1769074830.393150:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:65491:x1855007616993536:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 916us (1117us total) trans 8589944080 rc 0/0 00000100:00100000:1.0:1769074830.393158:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4724 00000020:00080000:1.0:1769074830.393163:0:6515:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1769074830.394851:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074830.394865:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.394877:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641520e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074830.394881:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.394890:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.394906:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x203:0x0].0x0 (ffff89b681697900) 00010000:00010000:3.0:1769074830.394909:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1769074830.394913:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 1/0,0 mode: --/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.394918:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b641520e00/0x7fe2323adfd54e90 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x203:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.394940:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074830.394942:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074830.394943:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074830.396035:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.396368:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616993792 00000100:00100000:2.0:1769074830.396378:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616993792 00000100:00100000:2.0:1769074830.396523:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4725 00000100:00100000:2.0:1769074830.396535:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:65491:x1855007616993792:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074830.396799:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:65491:x1855007616993792:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 265us (769us total) trans 8589944081 rc 0/0 00000100:00100000:2.0:1769074830.396835:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4725 00000100:00100000:0.0:1769074830.409653:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.409749:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616993920 00000100:00100000:2.0:1769074830.409759:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616993920 00000100:00100000:2.0:1769074830.409850:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13468 00000100:00100000:2.0:1769074830.409861:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:65492:x1855007616993920:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.409884:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.409904:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e97 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5ca expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.409971:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e9e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.409985:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e9e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.409997:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e9e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.410037:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074830.410042:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074830.410061:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e9e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.410086:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54e97 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ca expref: 31 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.410104:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e9e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ca expref: 30 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.410113:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.410116:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074830.410119:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074830.410188:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65492:x1855007616993920:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 328us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.410196:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13468 00000100:00100000:1.0:1769074830.413685:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.413991:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616994176 00000100:00100000:3.0:1769074830.414002:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616994176 00000100:00100000:3.0:1769074830.414061:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13469 00000100:00100000:3.0:1769074830.414071:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65492:x1855007616994176:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.414092:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.414114:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54ea5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5d1 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.414202:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414216:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414226:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414237:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414244:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414250:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414321:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414332:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414348:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414356:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414362:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.414366:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414373:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414384:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.414387:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.414391:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414397:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd54eac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414411:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414416:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414422:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414427:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.414430:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414436:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414456:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:3.0:1769074830.414459:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.414462:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414467:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54eb3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.414498:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54ea5 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5d1 expref: 31 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.414507:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.414509:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074830.414513:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba200/0x7fe2323adfd54ea5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5d1 expref: 31 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.414521:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba200, rc 0) 00000100:00100000:3.0:1769074830.414889:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65492:x1855007616994176:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 817us (1212us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074830.414897:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13469 00000100:00100000:1.0:1769074830.418836:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.418882:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616994304 00000100:00100000:3.0:1769074830.418892:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616994304 00000100:00100000:3.0:1769074830.418946:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13470 00000100:00100000:3.0:1769074830.418956:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65492:x1855007616994304:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074830.419031:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419048:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419059:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419071:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419077:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419083:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419155:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419164:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419170:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.419208:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:3.0:1769074830.419212:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074830.419224:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419238:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419248:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419254:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419261:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419267:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.419271:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419277:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419289:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.419291:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.419297:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419302:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54eba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419315:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419321:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419326:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419330:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.419333:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419350:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419358:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:3.0:1769074830.419361:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.419364:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419369:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ec1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419377:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419382:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419387:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419392:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.419394:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419400:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419406:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:3.0:1769074830.419409:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.419411:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.419420:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ec8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:3.0:1769074830.419510:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65492:x1855007616994304:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 554us (712us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.419518:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13470 00000100:00100000:0.0:1769074830.437473:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.437644:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616994560 00000100:00100000:2.0:1769074830.437655:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616994560 00000100:00100000:2.0:1769074830.437722:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13471 00000100:00100000:2.0:1769074830.437733:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65493:x1855007616994560:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.437753:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.437773:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54ecf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5d8 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.437874:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437887:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437897:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437906:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437913:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437918:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437986:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437992:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.437997:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438003:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438008:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.438011:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438018:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438028:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.438030:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.438034:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438040:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54ed6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438053:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438058:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438063:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438067:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.438069:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438073:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438078:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:2.0:1769074830.438080:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.438082:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438087:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54edd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.438111:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54ecf lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5d8 expref: 31 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.438119:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.438121:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.438125:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54ecf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5d8 expref: 31 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.438136:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074830.438200:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65493:x1855007616994560:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 469us (738us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074830.438208:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13471 00000100:00100000:1.0:1769074830.446932:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.447037:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616994688 00000100:00100000:3.0:1769074830.447047:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616994688 00000100:00100000:3.0:1769074830.447105:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13472 00000100:00100000:3.0:1769074830.447115:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65493:x1855007616994688:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074830.447200:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447216:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447227:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447238:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447245:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447252:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447324:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447333:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447369:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.447411:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:3.0:1769074830.447416:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074830.447429:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447452:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447460:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447465:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447472:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447477:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.447482:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447488:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447501:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.447504:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.447507:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447513:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54ee4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447526:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447532:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447538:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447542:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.447545:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447550:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447556:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:3.0:1769074830.447559:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.447562:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447566:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54eeb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447574:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447579:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447584:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447588:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.447592:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447597:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447603:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:3.0:1769074830.447606:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.447609:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.447614:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54ef2 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:3.0:1769074830.447702:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65493:x1855007616994688:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 587us (779us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.447711:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13472 00000100:00100000:1.0:1769074830.464538:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.464635:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616994944 00000100:00100000:2.0:1769074830.464647:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616994944 00000100:00100000:2.0:1769074830.464713:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13473 00000100:00100000:2.0:1769074830.464724:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65494:x1855007616994944:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.464745:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.464767:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54ef9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5df expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.464891:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.464909:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.464922:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.464933:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.464939:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.464945:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465030:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465037:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465042:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465048:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465053:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.465057:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465063:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465075:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.465077:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.465082:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465087:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f00 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465100:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465105:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465110:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465115:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.465117:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465123:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465128:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:2.0:1769074830.465130:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.465133:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465137:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f07 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.465173:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54ef9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5df expref: 31 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.465182:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.465184:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.465187:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54ef9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5df expref: 31 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.465196:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074830.465259:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65494:x1855007616994944:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 536us (730us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074830.465267:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13473 00000100:00100000:0.0:1769074830.468582:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.468673:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616995072 00000100:00100000:3.0:1769074830.468683:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616995072 00000100:00100000:3.0:1769074830.468749:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13474 00000100:00100000:3.0:1769074830.468759:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65494:x1855007616995072:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074830.468873:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.468889:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.468900:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.468910:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.468919:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.468924:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.468995:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469003:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469009:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074830.469046:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:3.0:1769074830.469051:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074830.469062:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469074:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469079:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469085:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469091:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469096:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.469100:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469107:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469117:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074830.469120:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.469124:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469130:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54f0e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469141:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469146:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469151:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469155:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febae00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.469157:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469163:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469169:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:3.0:1769074830.469171:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.469173:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469177:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54f15 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469185:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469190:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469195:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469200:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074830.469202:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469207:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469213:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:3.0:1769074830.469216:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074830.469219:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.469224:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f1c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:3.0:1769074830.469305:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65494:x1855007616995072:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 547us (731us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.469313:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13474 00000100:00100000:1.0:1769074830.489031:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.489146:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616995328 00000100:00100000:2.0:1769074830.489166:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616995328 00000100:00100000:2.0:1769074830.489229:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13475 00000100:00100000:2.0:1769074830.489241:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:65495:x1855007616995328:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.489263:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.489285:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f23 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5e6 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.489368:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489382:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489393:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489401:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489407:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489413:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489525:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54f38 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489533:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54f38 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489540:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54f38 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.489551:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:2.0:1769074830.489632:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489639:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489644:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489651:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489656:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.489661:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489668:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489680:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.489683:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.489688:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489694:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f2a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489706:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489726:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489732:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489740:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.489743:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489748:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489755:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:2.0:1769074830.489757:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.489760:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489765:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f31 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.489799:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5e6 expref: 33 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.489863:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54f38 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5e6 expref: 32 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.489870:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.489872:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074830.489875:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074830.489983:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:65495:x1855007616995328:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 744us (962us total) trans 8589944082 rc 0/0 00000100:00100000:2.0:1769074830.489992:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13475 00000100:00100000:1.0:1769074830.497013:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.497394:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616995584 00000100:00100000:2.0:1769074830.497405:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616995584 00000100:00100000:2.0:1769074830.497570:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13476 00000100:00100000:2.0:1769074830.497582:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:65495:x1855007616995584:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.497607:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.497611:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074830.497641:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54e9e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ca expref: 32 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.497661:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.497665:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074830.497679:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f46 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5ed expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.498967:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.498984:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.498995:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.499007:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x204:0x0]: returning size 0 00010000:00010000:2.0:1769074830.499057:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f46 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 34 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.499074:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 33 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.499083:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074830.499086:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074830.499089:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074830.499164:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:65495:x1855007616995584:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1583us (2160us total) trans 8589944083 rc 0/0 00000100:00100000:2.0:1769074830.499171:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13476 00000100:00100000:1.0:1769074830.502037:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.502842:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616995840 00000100:00100000:2.0:1769074830.502854:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616995840 00000100:00100000:2.0:1769074830.502963:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13477 00000100:00100000:2.0:1769074830.502974:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:65495:x1855007616995840:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.502998:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.503018:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f5b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5f4 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.503122:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.503137:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 34 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.503145:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.503180:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 34 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.503193:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 34 pid: 6514 timeout: 1918 lvb_type: 0 00000100:00100000:2.0:1769074830.503206:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625219072:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074830.503303:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8540 going to sleep for 11 seconds 00000100:00100000:1.0:1769074830.507275:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.507663:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6514:1855007625219072:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074830.507701:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074830.507888:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616995968 00000100:00100000:3.0:1769074830.507896:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616995968 00010000:00010000:3.0:1769074830.507931:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 35 pid: 6514 timeout: 1918 lvb_type: 0 00000100:00100000:3.0:1769074830.507977:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7198 00000100:00100000:3.0:1769074830.507986:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007616995968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074830.508001:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074830.508008:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 35 pid: 6514 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.508020:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 35 pid: 6514 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.508042:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd54f4d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5ed expref: 35 pid: 6514 timeout: 1918 lvb_type: 0 00010000:00010000:3.0:1769074830.508057:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:3.0:1769074830.508061:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074830.508069:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074830.508075:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074830.508134:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074830.508201:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:2409:x1855007616995968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 215us (936us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.508208:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7198 00010000:00010000:2.0:1769074830.509522:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.509535:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.509558:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x204:0x0]: returning size 0 00000004:00080000:2.0:1769074830.509944:0:6514:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c6:0x0], index 0: 0 00000004:00080000:2.0:1769074830.509954:0:6514:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2c8:0x0], index 1: 0 00010000:00010000:2.0:1769074830.510067:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510075:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510082:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510087:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.510091:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510099:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510113:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074830.510117:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074830.510121:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510131:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54f62 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.510280:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f5b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5f4 expref: 34 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.510291:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.510294:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.510298:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f5b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5f4 expref: 34 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.510308:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074830.510456:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:65495:x1855007616995840:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 7484us (8428us total) trans 8589944084 rc 301/301 00000100:00100000:2.0:1769074830.510466:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13477 00000100:00100000:1.0:1769074830.513378:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.513546:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616996224 00000100:00100000:2.0:1769074830.513558:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616996224 00000100:00100000:2.0:1769074830.513659:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4726 00000100:00100000:2.0:1769074830.513670:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:65495:x1855007616996224:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074830.513997:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:65495:x1855007616996224:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 328us (628us total) trans 8589944085 rc 0/0 00000100:00100000:2.0:1769074830.514004:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4726 00000100:00100000:1.0:1769074830.516726:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.517040:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616996352 00000100:00100000:3.0:1769074830.517050:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616996352 00000100:00100000:3.0:1769074830.517099:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14456 00000100:00100000:3.0:1769074830.517109:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2408:x1855007616996352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.517126:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.517146:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c5fb expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074830.517179:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c5:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1769074830.517198:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.517200:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.517209:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.517227:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.517240:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:3.0:1769074830.517348:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007616996352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 232us (621us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.517356:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14456 00000100:00100000:1.0:1769074830.520412:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.520509:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616996480 00000100:00100000:2.0:1769074830.520519:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616996480 00000100:00100000:2.0:1769074830.520611:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13478 00000100:00100000:2.0:1769074830.520621:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:65495:x1855007616996480:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.520649:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.520672:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f77 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c602 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.520908:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54f7e lrc: 3/1,0 mode: --/PR res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.520923:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54f7e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.520935:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54f7e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.520979:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x204:0x0]: returning size 0 00000004:00200000:2.0:1769074830.520984:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074830.521002:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54f7e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.521028:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f77 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c602 expref: 36 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.521046:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54f7e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c602 expref: 35 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.521055:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.521059:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1769074830.521062:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074830.521171:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:65495:x1855007616996480:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 551us (770us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.521178:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13478 00000100:00100000:1.0:1769074830.524376:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.524448:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616996608 00000100:00100000:0.0:1769074830.524457:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616996608 00000100:00100000:0.0:1769074830.524508:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14457 00000100:00100000:0.0:1769074830.524516:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007616996608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074830.524532:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074830.524553:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c609 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074830.524575:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c6:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1769074830.524591:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.524593:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.524601:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 8 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074830.524616:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 8 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074830.524628:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164800, rc 0) 00000100:00100000:0.0:1769074830.524681:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616996608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 166us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.524696:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14457 00000100:00100000:1.0:1769074830.526349:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.526424:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616996736 00000100:00100000:3.0:1769074830.526433:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616996736 00000100:00100000:3.0:1769074830.526482:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14458 00000100:00100000:3.0:1769074830.526492:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007616996736:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.526510:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.526530:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c610 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074830.526557:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1769074830.526575:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.526577:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.526585:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.526600:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.526611:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:3.0:1769074830.526667:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616996736:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 177us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.526675:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14458 00000100:00100000:1.0:1769074830.530652:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.530764:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616996992 00000100:00100000:2.0:1769074830.530776:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616996992 00000100:00100000:2.0:1769074830.530873:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13479 00000100:00100000:2.0:1769074830.530883:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:65495:x1855007616996992:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.530903:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.530925:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f93 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c617 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.531028:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f9a lrc: 3/0,1 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.531045:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f9a lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.531056:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f9a lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.531068:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00010000:00010000:2.0:1769074830.531185:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd54f93 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c617 expref: 38 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.531204:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f9a lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c617 expref: 37 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.531212:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.531215:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074830.531218:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074830.531281:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:65495:x1855007616996992:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 399us (638us total) trans 8589944086 rc 0/0 00000100:00100000:2.0:1769074830.531288:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13479 00000100:00100000:1.0:1769074830.535595:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.535699:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616997120 00000100:00100000:0.0:1769074830.535709:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616997120 00000100:00100000:0.0:1769074830.535758:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14459 00000100:00100000:0.0:1769074830.535767:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616997120:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:0.0:1769074830.535838:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535852:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.535854:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.535859:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535872:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535894:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535902:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535910:0:36516:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535917:0:36516:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074830.535921:0:36516:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535929:0:36516:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535947:0:36516:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.535954:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b682164200/0x7fe2323adfd54fa8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074830.536012:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616997120:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 246us (426us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.536018:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14459 00000100:00100000:1.0:1769074830.575909:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.576000:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616997248 00000100:00100000:3.0:1769074830.576010:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616997248 00000100:00100000:3.0:1769074830.576070:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13480 00000100:00100000:3.0:1769074830.576079:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+37:65495:x1855007616997248:12345-192.168.204.60@tcp:44:lfs.0 00000001:00080000:0.0:1769074830.578045:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944086 is committed 00000004:00080000:0.0:1769074830.578066:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1963 committed 00000100:00100000:1.0:1769074830.578375:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074830.578392:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625219200:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1769074830.578458:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1769074830.578570:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.578582:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.578590:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.578596:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbe00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.578600:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.578607:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.578623:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690600) 00010000:00010000:2.0:1769074830.578627:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.578632:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.578638:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd54da2 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:2.0:1769074830.578890:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625219200 00000100:00100000:2.0:1769074830.578898:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625219200 00000100:00100000:2.0:1769074830.578953:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14460 00000100:00100000:2.0:1769074830.578962:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625219200:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1769074830.578988:0:28519:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:711 count 1 00010000:00010000:2.0:1769074830.579010:0:28519:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.579029:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074830.579045:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074830.579047:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074830.579051:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.579079:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 7 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074830.579092:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 7 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:2.0:1769074830.579104:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:28519:1855007625219328:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074830.579181:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b2b8780 going to sleep for 11 seconds 00000100:00080000:2.0:1769074830.579194:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074830.583124:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:65495:x1855007616997248:12345-192.168.204.60@tcp:44:lfs.0 Request processed in 7034us (7212us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.583137:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13480 00000100:00100000:3.0:1769074830.584280:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:28519:1855007625219328:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074830.584312:0:28519:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074830.587518:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.587642:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616997376 00000100:00100000:2.0:1769074830.587652:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616997376 00010000:00010000:2.0:1769074830.587774:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 8 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:2.0:1769074830.587882:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7199 00000100:00100000:2.0:1769074830.587893:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007616997376:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074830.587910:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074830.587917:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 8 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:2.0:1769074830.587931:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 8 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:2.0:1769074830.587956:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb200/0x7fe2323adfd54dfd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c592 expref: 8 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:2.0:1769074830.587973:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074830.587975:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074830.587980:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.587990:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.588017:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074830.588069:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007616997376:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 178us (561us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.588077:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7199 00010000:00010000:0.0:1769074830.588134:0:28519:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.588145:0:28519:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.588164:0:28519:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.588172:0:28519:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.588194:0:28519:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1769074830.588455:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625219200:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9494us (9996us total) trans 17179874610 rc 0/0 00000100:00100000:0.0:1769074830.588467:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14460 00000100:00100000:0.0:1769074830.589065:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.589137:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.589172:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616997504 00000100:00100000:0.0:1769074830.589179:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616997504 00000100:00100000:0.0:1769074830.589207:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616997632 00000100:00100000:0.0:1769074830.589210:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616997632 00000100:00100000:0.0:1769074830.589228:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14461 00000100:00100000:0.0:1769074830.589235:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616997504:12345-192.168.204.60@tcp:16:lfs.0 00000100:00100000:0.0:1769074830.589308:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616997504:12345-192.168.204.60@tcp:16:lfs.0 Request processed in 73us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.589315:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14461 00000100:00100000:0.0:1769074830.589329:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14462 00000100:00100000:0.0:1769074830.589347:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007616997632:12345-192.168.204.60@tcp:16:lfs.0 00000100:00100000:0.0:1769074830.589386:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007616997632:12345-192.168.204.60@tcp:16:lfs.0 Request processed in 52us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.589392:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14462 00000004:00080000:0.0:1769074830.590097:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759ad80/2, rc 0, transno 17179874610 00000100:00100000:0.0:1769074830.590108:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625219200:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1769074830.593981:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074830.593996:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.594016:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074830.594021:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.594032:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.594055:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.594063:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd54faf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.594087:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074830.594089:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074830.594091:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074830.605086:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.605546:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616997760 00000100:00100000:2.0:1769074830.605557:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616997760 00000100:00100000:2.0:1769074830.605911:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14463 00000100:00100000:2.0:1769074830.605923:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007616997760:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:2.0:1769074830.605966:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.605987:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074830.605990:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074830.605995:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606010:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606037:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606045:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606054:0:36516:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606061:0:36516:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074830.606066:0:36516:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606074:0:36516:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606088:0:36516:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.606101:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89800/0x7fe2323adfd54fb6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074830.606230:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007616997760:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 309us (1153us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.606239:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14463 00000100:00100000:0.0:1769074830.609127:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.609299:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998016 00000100:00100000:2.0:1769074830.609309:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998016 00000100:00100000:2.0:1769074830.609411:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13481 00000100:00100000:2.0:1769074830.609420:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:65495:x1855007616998016:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074830.609552:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074830.609578:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd54f7e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x204:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c602 expref: 12 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.609595:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.609605:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074830.609634:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,1 mode: --/PW res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.609645:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.609655:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00010000:2.0:1769074830.609996:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008f43b1a6 reply state 00000000133b2f33 transno 8589944087 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.610012:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.610018:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074830.610021:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074830.610023:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074830.610165:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:65495:x1855007616998016:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 736us (1039us total) trans 8589944087 rc 0/0 00000100:00100000:2.0:1769074830.610173:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13481 00000100:00100000:1.0:1769074830.612663:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.612924:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998144 00000100:00100000:2.0:1769074830.612934:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998144 00000100:00100000:2.0:1769074830.613021:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13482 00000100:00100000:2.0:1769074830.613031:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:65495:x1855007616998144:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.613053:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.613073:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c61e expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.613123:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.613138:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.613167:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074830.613171:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074830.613174:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074830.613232:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:65495:x1855007616998144:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 202us (578us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.613242:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13482 00000100:00100000:1.0:1769074830.615625:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.615722:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998272 00000100:00100000:3.0:1769074830.615732:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998272 00000100:00100000:3.0:1769074830.615792:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14464 00000100:00100000:3.0:1769074830.615801:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:65495:x1855007616998272:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.615838:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.615855:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54fcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c625 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.615883:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.615898:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.615900:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.615904:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54fcb lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c625 expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.615938:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.615951:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 9 pid: 36516 timeout: 1918 lvb_type: 1 00000100:00100000:3.0:1769074830.615965:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625219456:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074830.616036:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266d80 going to sleep for 11 seconds 00000100:00100000:1.0:1769074830.619357:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.619580:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998400 00000100:00100000:0.0:1769074830.619591:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998400 00010000:00010000:0.0:1769074830.619627:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 10 pid: 36516 timeout: 1918 lvb_type: 1 00000100:00100000:0.0:1769074830.619670:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7200 00000100:00100000:0.0:1769074830.619677:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007616998400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074830.619699:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074830.619719:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 10 pid: 36516 timeout: 1918 lvb_type: 1 00010000:00010000:0.0:1769074830.619731:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 10 pid: 36516 timeout: 1918 lvb_type: 1 00010000:00010000:0.0:1769074830.619754:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.619756:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.619762:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54fcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c625 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.619771:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54fcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c625 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.619822:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54fcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c625 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074830.619836:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625219584:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074830.619899:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093e40 going to sleep for 11 seconds 00010000:00010000:3.0:1769074830.620329:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b682164800/0x7fe2323adfd54f85 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c609 expref: 10 pid: 36516 timeout: 1918 lvb_type: 1 00000100:00100000:3.0:1769074830.620353:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625219456:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074830.620379:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb800/0x7fe2323adfd54fcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c625 expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.620389:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb800, rc 0) 00000100:00100000:3.0:1769074830.620421:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:65495:x1855007616998272:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4621us (4805us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.620429:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14464 00000100:00100000:0.0:1769074830.624999:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625219584:192.168.204.60@tcp:105: 00010000:00010000:0.0:1769074830.625024:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074830.625436:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616998400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5756us (6096us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.625448:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7200 00000100:00100000:1.0:1769074830.628080:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.628250:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998528 00000100:00100000:3.0:1769074830.628261:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998528 00000100:00100000:3.0:1769074830.628356:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14465 00000100:00100000:3.0:1769074830.628366:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:65495:x1855007616998528:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.628383:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.628400:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba200/0x7fe2323adfd54fd2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c62c expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.628430:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.628439:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.628441:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.628446:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba200/0x7fe2323adfd54fd2 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c62c expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.628479:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 8 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074830.628491:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 8 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:3.0:1769074830.628504:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625219712:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074830.628588:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266d80 going to sleep for 11 seconds 00000100:00100000:2.0:1769074830.630944:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.631032:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998656 00000100:00100000:0.0:1769074830.631040:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998656 00010000:00010000:0.0:1769074830.631075:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 9 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:0.0:1769074830.631123:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7201 00000100:00100000:0.0:1769074830.631133:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007616998656:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074830.631147:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074830.631177:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 9 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:0.0:1769074830.631189:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 9 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:0.0:1769074830.631212:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.631215:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074830.631220:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba200/0x7fe2323adfd54fd2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c62c expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.631231:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba200/0x7fe2323adfd54fd2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c62c expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.631266:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b67feba200/0x7fe2323adfd54fd2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c62c expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074830.631284:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625219840:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074830.631368:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093e40 going to sleep for 11 seconds 00010000:00010000:3.0:1769074830.631583:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febae00/0x7fe2323adfd54f8c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c610 expref: 9 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:3.0:1769074830.631602:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625219712:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074830.631626:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba200/0x7fe2323adfd54fd2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c62c expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.631635:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba200, rc 0) 00000100:00100000:3.0:1769074830.631665:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:65495:x1855007616998528:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 3301us (3594us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.631673:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14465 00000100:00100000:0.0:1769074830.635031:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.635067:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.635123:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998784 00000100:00100000:3.0:1769074830.635132:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998784 00000100:00100000:3.0:1769074830.635197:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616998912 00000100:00100000:3.0:1769074830.635200:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616998912 00000100:00100000:3.0:1769074830.635221:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14466 00000100:00100000:3.0:1769074830.635229:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616998784:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:3.0:1769074830.635404:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c6:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1769074830.635408:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c6:0x0] updating lvb mtime from disk: 0 -> 1769074829 00002000:00010000:3.0:1769074830.635410:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c6:0x0] updating lvb atime from disk: 0 -> 1769074829 00002000:00010000:3.0:1769074830.635413:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c6:0x0] updating lvb ctime from disk: 0 -> 1769074830 00002000:00010000:3.0:1769074830.635415:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c6:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1769074830.635667:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007616998784:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 437us (646us total) trans 21474841286 rc 0/0 00000100:00100000:3.0:1769074830.635675:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14466 00000100:00100000:3.0:1769074830.635785:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14467 00000100:00100000:3.0:1769074830.635792:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007616998912:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:3.0:1769074830.635897:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c8:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1769074830.635900:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c8:0x0] updating lvb mtime from disk: 0 -> 1769074829 00002000:00010000:3.0:1769074830.635902:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c8:0x0] updating lvb atime from disk: 0 -> 1769074829 00002000:00010000:3.0:1769074830.635904:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c8:0x0] updating lvb ctime from disk: 0 -> 1769074830 00002000:00010000:3.0:1769074830.635906:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c8:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1769074830.635932:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007616998912:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 140us (863us total) trans 17179874611 rc 0/0 00000100:00100000:3.0:1769074830.635937:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14467 00000100:00100000:1.0:1769074830.636091:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8424:1855007625219840:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074830.636111:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074830.636136:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007616998656:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 5004us (5202us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074830.636144:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7201 00000100:00100000:1.0:1769074830.639257:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.639377:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616999168 00000100:00100000:2.0:1769074830.639388:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616999168 00000100:00100000:2.0:1769074830.639556:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13483 00000100:00100000:2.0:1769074830.639569:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:65495:x1855007616999168:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074830.639622:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074830.639652:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd54f38 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5e6 expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.639670:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.639676:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074830.639734:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.639748:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.639757:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00010000:2.0:1769074830.640036:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000c536b246 reply state 0000000084609100 transno 8589944088 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.640049:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.640056:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.640059:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074830.640061:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074830.640290:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:65495:x1855007616999168:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 721us (1039us total) trans 8589944088 rc 0/0 00000100:00100000:2.0:1769074830.640298:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13483 00000100:00100000:0.0:1769074830.642913:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.643119:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616999296 00000100:00100000:1.0:1769074830.643131:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616999296 00000100:00100000:1.0:1769074830.643265:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14468 00000100:00100000:1.0:1769074830.643275:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:65495:x1855007616999296:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074830.643292:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074830.643311:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b641521000/0x7fe2323adfd54fe0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c633 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074830.643352:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 10 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074830.643361:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074830.643364:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074830.643368:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641521000/0x7fe2323adfd54fe0 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c633 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074830.643406:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 10 pid: 28519 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074830.643419:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 10 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:1.0:1769074830.643432:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625219968:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074830.643605:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3e40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074830.649133:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.649247:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625219968:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074830.649286:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b641521000/0x7fe2323adfd54fe0 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c633 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.649308:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641521000, rc 0) 00000100:00100000:3.0:1769074830.649418:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616999424 00000100:00100000:3.0:1769074830.649426:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616999424 00010000:00010000:3.0:1769074830.649475:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 11 pid: 28519 timeout: 1918 lvb_type: 1 00000100:00100000:3.0:1769074830.649509:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7202 00000100:00100000:3.0:1769074830.649519:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007616999424:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074830.649534:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074830.649556:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 11 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:3.0:1769074830.649569:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 11 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:3.0:1769074830.649593:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54f70 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5fb expref: 11 pid: 28519 timeout: 1918 lvb_type: 1 00010000:00010000:3.0:1769074830.649607:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.649610:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074830.649615:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b641521000/0x7fe2323adfd54fe0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c633 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.649625:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b641521000/0x7fe2323adfd54fe0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c633 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.649656:0:8424:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b641521000/0x7fe2323adfd54fe0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c633 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074830.649669:0:8424:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625220096:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074830.650102:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:65495:x1855007616999296:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6825us (7197us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.650111:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14468 00000100:00100000:3.0:1769074830.650280:0:8424:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266d80 going to sleep for 11 seconds 00000100:00100000:0.0:1769074830.654224:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.654291:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616999552 00000100:00100000:2.0:1769074830.654301:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616999552 00000100:00100000:2.0:1769074830.654371:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14469 00000100:00100000:2.0:1769074830.654380:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007616999552:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:2.0:1769074830.654547:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1769074830.654550:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c5:0x0] updating lvb mtime from disk: 0 -> 1769074829 00002000:00010000:2.0:1769074830.654553:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c5:0x0] updating lvb atime from disk: 0 -> 1769074829 00002000:00010000:2.0:1769074830.654555:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c5:0x0] updating lvb ctime from disk: 0 -> 1769074830 00002000:00010000:2.0:1769074830.654557:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c5:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074830.654622:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2406:x1855007616999552:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 242us (410us total) trans 21474841287 rc 0/0 00000100:00100000:2.0:1769074830.654631:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14469 00000100:00100000:3.0:1769074830.658140:0:8424:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8424:1855007625220096:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074830.658179:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074830.658413:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.658788:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616999808 00000100:00100000:3.0:1769074830.658794:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616999808 00000100:00100000:3.0:1769074830.658857:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4727 00000100:00100000:3.0:1769074830.658863:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:65495:x1855007616999808:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:3.0:1769074830.658931:0:6515:0:(mdt_open.c:2313:mdt_close_handle_layouts()) ### [0x200000405:0x202:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f9a lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c617 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.658963:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.658973:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 15 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.658979:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.659003:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 15 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.659012:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 15 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:3.0:1769074830.659023:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625220224:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074830.659092:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266d80 going to sleep for 11 seconds 00000100:00100000:3.0:1769074830.662865:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007616999424:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13332us (13726us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.662882:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7202 00000100:00100000:3.0:1769074830.663285:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625220224:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074830.663311:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:1.0:1769074830.664175:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.664952:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007616999936 00000100:00100000:2.0:1769074830.664962:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007616999936 00010000:00010000:2.0:1769074830.665003:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 16 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:2.0:1769074830.665090:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7203 00000100:00100000:2.0:1769074830.665100:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:2408:x1855007616999936:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074830.665120:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074830.665126:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 16 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:2.0:1769074830.665139:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 16 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:2.0:1769074830.665175:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febac00/0x7fe2323adfd54e89 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c5c3 expref: 16 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:2.0:1769074830.665192:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.665197:0:8424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.665207:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.665215:0:8424:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074830.665244:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074830.665298:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2408:x1855007616999936:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 198us (1128us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.665306:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7203 00010000:00010000:3.0:1769074830.665405:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.665413:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.665448:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.665460:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.665468:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 4/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.665502:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 14 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.665515:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 14 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:3.0:1769074830.665529:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625220352:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074830.665590:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266d80 going to sleep for 11 seconds 00000100:00100000:3.0:1769074830.668731:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.669073:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625220352:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074830.669100:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:0.0:1769074830.669257:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617000064 00000100:00100000:0.0:1769074830.669265:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617000064 00010000:00010000:0.0:1769074830.669303:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 15 pid: 6513 timeout: 1918 lvb_type: 0 00000100:00100000:0.0:1769074830.669353:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7204 00000100:00100000:0.0:1769074830.669362:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:2409:x1855007617000064:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074830.669379:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074830.669386:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 15 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:0.0:1769074830.669398:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 15 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:0.0:1769074830.669420:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd54fc4 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x204:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c61e expref: 15 pid: 6513 timeout: 1918 lvb_type: 0 00010000:00010000:0.0:1769074830.669651:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:0.0:1769074830.669660:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074830.669671:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074830.669678:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:0.0:1769074830.669832:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074830.669892:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2409:x1855007617000064:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 529us (1169us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.669899:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7204 00010000:00010000:3.0:1769074830.669966:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:3.0:1769074830.669973:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:3.0:1769074830.670279:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220480:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1769074830.670330:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074830.670375:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5492666c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074830.673111:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625220480 00000100:00100000:0.0:1769074830.673126:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625220480 00000100:00100000:0.0:1769074830.673201:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2696 00000100:00100000:0.0:1769074830.673211:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625220480:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.673365:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625220480:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 155us (3034us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.673372:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2696 00000100:00100000:0.0:1769074830.673426:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220480:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.673727:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220608:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.673767:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074830.673790:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b676093f00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074830.673832:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625220608 00000100:00100000:0.0:1769074830.673837:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625220608 00000100:00100000:0.0:1769074830.673865:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2697 00000100:00100000:0.0:1769074830.673871:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625220608:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.673927:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625220608:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 55us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.673934:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2697 00000100:00100000:0.0:1769074830.673965:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220608:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.674017:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220736:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.674044:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074830.674059:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64995ad80 going to sleep for 11 seconds 00000100:00100000:0.0:1769074830.674071:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625220736 00000100:00100000:0.0:1769074830.674075:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625220736 00000100:00100000:0.0:1769074830.674097:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2698 00000100:00100000:0.0:1769074830.674104:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6515:x1855007625220736:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.674172:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6515:x1855007625220736:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 68us (127us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.674178:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2698 00000100:00100000:0.0:1769074830.674212:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220736:0@lo:1000:mdt_rdpg00_000.0 00000040:00080000:0.0:1769074830.674387:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].83, 280 off86304 00000001:00080000:0.0:1769074830.674400:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944089 00000100:00100000:0.0:1769074830.676209:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220864:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.676274:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074830.676307:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64ee50a80 going to sleep for 11 seconds 00000100:00100000:3.0:1769074830.677627:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625220864 00000100:00100000:3.0:1769074830.677641:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625220864 00000100:00100000:3.0:1769074830.677935:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2699 00000100:00100000:3.0:1769074830.677947:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6515:x1855007625220864:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1769074830.678135:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6515:x1855007625220864:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 190us (1859us total) trans 17179874612 rc 0/0 00000100:00100000:3.0:1769074830.678142:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2699 00000100:00100000:1.0:1769074830.678238:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220864:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:1.0:1769074830.678297:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220992:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:1.0:1769074830.678366:0:6515:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074830.678622:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b6f3840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074830.678847:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625220992 00000100:00100000:0.0:1769074830.678853:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625220992 00000100:00100000:0.0:1769074830.678897:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2700 00000100:00100000:0.0:1769074830.678904:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6515:x1855007625220992:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769074830.678996:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6515:x1855007625220992:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 93us (661us total) trans 21474841288 rc 0/0 00000100:00100000:0.0:1769074830.679003:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2700 00000100:00100000:1.0:1769074830.679082:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:6515:1855007625220992:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:1.0:1769074830.679147:0:6515:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944090 00010000:00010000:1.0:1769074830.679193:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679202:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679211:0:6515:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679217:0:6515:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074830.679222:0:6515:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679229:0:6515:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679243:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:1.0:1769074830.679247:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1769074830.679251:0:6515:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 2/0,0 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679256:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd54fee lrc: 0/0,0 mode: --/EX res: [0x200000405:0x204:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679268:0:6515:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679273:0:6515:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679277:0:6515:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679281:0:6515:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074830.679283:0:6515:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679288:0:6515:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679298:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:1.0:1769074830.679300:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1769074830.679303:0:6515:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679308:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd54fe7 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x202:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074830.679318:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:1.0:1769074830.679321:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074830.679327:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd54f9a lrc: 0/0,0 mode: --/PW res: [0x200000405:0x202:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c617 expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074830.683038:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:65495:x1855007616999808:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 24159us (24611us total) trans 8589944091 rc 0/0 00000100:00100000:1.0:1769074830.683053:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4727 00000020:00080000:1.0:1769074830.683062:0:6515:0:(tgt_main.c:782:tgt_ses_key_exit()) total 3 transactions per RPC 00000100:00100000:0.0:1769074830.687394:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.687893:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617000192 00000100:00100000:3.0:1769074830.687904:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617000192 00000100:00100000:3.0:1769074830.687953:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13484 00000100:00100000:3.0:1769074830.687963:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:65495:x1855007617000192:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.687985:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.688005:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ff5 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c63a expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.688061:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ff5 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c63a expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.688076:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febae00/0x7fe2323adfd54ff5 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x202:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c63a expref: 16 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.688094:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:3.0:1769074830.688098:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074830.688101:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febae00, rc 0) 00000100:00100000:3.0:1769074830.688274:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:65495:x1855007617000192:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 312us (890us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074830.688281:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13484 00000100:00100000:0.0:1769074830.699605:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074830.702333:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617000448 00000100:00100000:1.0:1769074830.702358:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617000448 00000100:00100000:1.0:1769074830.702419:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4728 00000100:00100000:1.0:1769074830.702428:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:65495:x1855007617000448:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074830.703483:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:65495:x1855007617000448:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 1048us (3881us total) trans 8589944092 rc 0/0 00000100:00100000:1.0:1769074830.703493:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4728 00000100:00100000:1.0:1769074830.710237:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.710536:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617000704 00000100:00100000:0.0:1769074830.710547:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617000704 00000100:00100000:0.0:1769074830.710604:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4729 00000100:00100000:0.0:1769074830.710613:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:65495:x1855007617000704:12345-192.168.204.60@tcp:35:lfs.0 00000004:00080000:0.0:1769074830.710845:0:15947:0:(mdd_object.c:3993:mdd_close()) Object [0x200000405:0x204:0x0] is deleted from orphan list, OSS objects to be destroyed. 00010000:00010000:0.0:1769074830.710926:0:15947:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 3/0,1 mode: --/PW res: [0x200000405:0x204:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.710944:0:15947:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.710956:0:15947:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.710964:0:15947:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.710969:0:15947:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.710974:0:15947:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 15947 initiator: MDT0 00010000:00010000:0.0:1769074830.710999:0:15947:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074830.711424:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074830.711431:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074830.711438:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074830.711442:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074830.711449:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074830.711461:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:3.0:1769074830.711464:0:6502:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1769074830.711469:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 1/0,0 mode: --/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074830.711474:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165a00/0x7fe2323adfd54ffc lrc: 0/0,0 mode: --/PW res: [0x200000405:0x204:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 15947 initiator: MDT0 00010000:00010000:3.0:1769074830.711486:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074830.711487:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074830.711489:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074830.711526:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:65495:x1855007617000704:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 913us (1297us total) trans 8589944093 rc 0/0 00000100:00100000:0.0:1769074830.711534:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4729 00000100:00100000:2.0:1769074830.738763:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.738878:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617000832 00000100:00100000:0.0:1769074830.738888:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617000832 00000100:00100000:0.0:1769074830.738949:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13485 00000100:00100000:0.0:1769074830.738960:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:65496:x1855007617000832:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074830.738985:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074830.739007:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55003 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c641 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.739076:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd5500a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.739090:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd5500a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.739102:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd5500a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074830.739165:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1769074830.739169:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074830.739189:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd5500a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.739215:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55003 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c641 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.739233:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd5500a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c641 expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074830.739241:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074830.739244:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769074830.739248:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164000, rc 0) 00000100:00100000:0.0:1769074830.739315:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65496:x1855007617000832:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 357us (562us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.739323:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13485 00000100:00100000:2.0:1769074830.743421:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074830.743572:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617001088 00000100:00100000:3.0:1769074830.743582:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617001088 00000100:00100000:3.0:1769074830.743639:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13486 00000100:00100000:3.0:1769074830.743649:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65496:x1855007617001088:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074830.743669:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074830.743695:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55011 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c648 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074830.743768:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.743781:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074830.743792:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744186:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744196:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744204:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744288:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744296:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744302:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744308:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744313:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.744317:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744324:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744347:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.744351:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.744355:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744361:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55018 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744377:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744383:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744388:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744392:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.744395:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744399:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744406:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b681690f00) 00010000:00010000:2.0:1769074830.744408:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.744411:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744415:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd5501f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.744511:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55011 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c648 expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.744520:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.744522:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.744526:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55011 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c648 expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.744539:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:2.0:1769074830.744637:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65496:x1855007617001088:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 989us (1226us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074830.744646:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13486 00000100:00100000:1.0:1769074830.748934:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.749106:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617001216 00000100:00100000:2.0:1769074830.749116:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617001216 00000100:00100000:2.0:1769074830.749176:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13487 00000100:00100000:2.0:1769074830.749186:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65496:x1855007617001216:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074830.749255:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749270:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749281:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749295:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749303:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749309:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749386:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749394:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749400:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074830.749437:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:2.0:1769074830.749452:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074830.749466:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749478:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749484:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749490:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749496:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749502:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.749506:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749513:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749524:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.749527:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.749532:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749537:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55026 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749552:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749558:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749563:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749568:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.749570:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749576:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749582:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.749585:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.749588:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749593:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5502d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749605:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749610:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749615:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749620:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.749623:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749629:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749635:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.749637:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.749639:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.749644:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55034 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:2.0:1769074830.749727:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65496:x1855007617001216:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 541us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.749736:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13487 00010000:00010000:1.0:1769074830.750192:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074830.750983:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074830.751001:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625221120:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074830.751052:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074830.751177:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221120 00000100:00100000:0.0:1769074830.751184:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221120 00000100:00100000:0.0:1769074830.751219:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1212 00000100:00100000:0.0:1769074830.751227:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625221120:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074830.751252:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:0.0:1769074830.751296:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625221120:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.751303:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1212 00000100:00100000:0.0:1769074830.751320:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074830.751328:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625221248:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074830.751365:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074830.751392:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221248 00000100:00100000:0.0:1769074830.751396:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221248 00000100:00100000:0.0:1769074830.751430:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1213 00000100:00100000:0.0:1769074830.751436:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625221248:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074830.751459:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:0.0:1769074830.751489:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625221248:12345-0@lo:13:osp-pre-0-1.0 Request processed in 52us (123us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.751495:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1213 00000100:00080000:0.0:1769074830.751510:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074830.751543:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625221248:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074830.751573:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625221120:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074830.752536:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074830.754711:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625221376:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074830.754784:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074830.754988:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221376 00000100:00100000:1.0:1769074830.754997:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221376 00000100:00100000:1.0:1769074830.755048:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1214 00000100:00100000:1.0:1769074830.755058:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625221376:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074830.755084:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 8697152 pending 0 free 3917004800 avail 3685482496 00000100:00100000:1.0:1769074830.755138:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625221376:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074830.755145:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1214 00000100:00100000:2.0:1769074830.755407:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625221376:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074830.768993:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.769111:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617001472 00000100:00100000:2.0:1769074830.769122:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617001472 00000100:00100000:2.0:1769074830.769195:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13488 00000100:00100000:2.0:1769074830.769204:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65497:x1855007617001472:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.769224:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.769245:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5503b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c64f expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.769320:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769345:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769358:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769374:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769381:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769387:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769469:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769477:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769483:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769489:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769495:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.769499:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769506:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769517:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.769520:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.769524:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769529:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55042 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769542:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769546:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769551:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769555:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.769557:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769562:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769569:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.769572:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.769575:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769579:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55049 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.769613:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5503b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c64f expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.769622:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.769624:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.769628:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5503b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c64f expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.769637:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:2.0:1769074830.769703:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65497:x1855007617001472:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 500us (720us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074830.769711:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13488 00000100:00100000:2.0:1769074830.774905:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.774993:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617001600 00000100:00100000:0.0:1769074830.775004:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617001600 00000100:00100000:0.0:1769074830.775063:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13489 00000100:00100000:0.0:1769074830.775072:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65497:x1855007617001600:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074830.775143:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775170:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775181:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775194:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775201:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775207:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775285:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775293:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775299:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:0.0:1769074830.775355:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:0.0:1769074830.775360:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074830.775372:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775385:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775394:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775399:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775405:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775410:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.775413:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775422:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775459:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074830.775464:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.775471:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775476:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55050 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775489:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775495:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775500:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775505:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.775507:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775513:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775520:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b643a80100) 00010000:00010000:0.0:1769074830.775524:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.775527:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775534:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd55057 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775552:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775560:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775566:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775571:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.775573:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775578:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775585:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:0.0:1769074830.775587:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.775590:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074830.775599:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165200/0x7fe2323adfd5505e lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:0.0:1769074830.775708:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65497:x1855007617001600:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 632us (809us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.775716:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13489 00000100:00100000:1.0:1769074830.794946:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.795127:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617001856 00000100:00100000:2.0:1769074830.795137:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617001856 00000100:00100000:2.0:1769074830.795199:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13490 00000100:00100000:2.0:1769074830.795208:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65498:x1855007617001856:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.795228:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.795250:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55065 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c656 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.795323:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795350:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795363:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795378:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795384:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795390:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795478:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795485:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795491:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795496:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795502:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.795505:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795513:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795523:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.795527:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.795531:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795537:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5506c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795550:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795555:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795560:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795564:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.795567:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795571:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795577:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.795579:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.795582:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795586:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55073 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.795616:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55065 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c656 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.795625:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.795627:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.795630:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd55065 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c656 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.795642:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:2.0:1769074830.796020:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65498:x1855007617001856:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 811us (1087us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074830.796029:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13490 00000100:00100000:1.0:1769074830.801495:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.801674:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617001984 00000100:00100000:2.0:1769074830.801693:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617001984 00000100:00100000:2.0:1769074830.801745:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13491 00000100:00100000:2.0:1769074830.801755:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65498:x1855007617001984:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074830.801852:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801869:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801879:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801893:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801899:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801904:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801975:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801983:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.801990:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074830.802030:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:2.0:1769074830.802035:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074830.802048:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802062:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802068:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802074:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802080:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802086:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.802090:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802096:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802108:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.802111:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.802116:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802121:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5507a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802134:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802139:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802144:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802161:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.802164:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802169:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802176:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.802179:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.802181:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802186:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55081 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802198:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802203:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802208:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802212:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.802215:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802220:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802226:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074830.802228:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.802231:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074830.802236:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55088 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074830.802298:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65498:x1855007617001984:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 544us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074830.802306:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13491 00000100:00100000:0.0:1769074830.817118:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074830.817230:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617002240 00000100:00100000:2.0:1769074830.817242:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617002240 00000100:00100000:2.0:1769074830.817306:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13492 00000100:00100000:2.0:1769074830.817316:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65499:x1855007617002240:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074830.817347:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074830.817368:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5508f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c65d expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.817456:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817471:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817482:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817498:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817504:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817510:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817586:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817593:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817599:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817605:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817611:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.817615:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817622:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817632:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.817635:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.817640:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817646:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55096 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817661:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817667:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817672:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817676:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074830.817691:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817697:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817704:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b64effac00) 00010000:00010000:2.0:1769074830.817706:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074830.817708:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817714:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5509d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074830.817746:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5508f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c65d expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.817755:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074830.817757:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074830.817761:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd5508f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c65d expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074830.817775:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88800, rc 0) 00000100:00100000:2.0:1769074830.817876:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65499:x1855007617002240:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 561us (767us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074830.817884:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13492 00000100:00100000:1.0:1769074830.823080:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074830.823195:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617002368 00000100:00100000:0.0:1769074830.823205:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617002368 00000100:00100000:0.0:1769074830.823272:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13493 00000100:00100000:0.0:1769074830.823281:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65499:x1855007617002368:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074830.823367:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823383:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823394:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823408:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823414:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823419:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823509:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823522:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823530:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074830.823568:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x202:0x0]: returning size 0 00000004:00200000:0.0:1769074830.823573:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074830.823585:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823598:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823605:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823611:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823617:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823622:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.823627:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823633:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823645:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074830.823648:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.823653:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823659:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.823674:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824157:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824164:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824169:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.824172:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824178:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824188:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xdeaaee82 (ffff89b643a80100) 00010000:00010000:0.0:1769074830.824190:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.824194:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824200:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550ab lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xdeaaee82 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824214:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824220:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824226:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824232:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074830.824235:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824241:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824248:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:0.0:1769074830.824251:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074830.824254:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074830.824259:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550b2 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074830.824353:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:65499:x1855007617002368:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1070us (1282us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074830.824361:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13493 00000100:00100000:0.0:1769074831.261609:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625221504:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074831.261680:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074831.261782:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221504 00000100:00100000:1.0:1769074831.261790:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221504 00000100:00100000:1.0:1769074831.261888:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1215 00000100:00100000:1.0:1769074831.261897:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625221504:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1769074831.261919:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 8697152 pending 0 free 3917004800 avail 3685482496 00000100:00080000:2.0:1769074831.261985:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074831.264581:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625221504:12345-0@lo:13:osp-pre-1-1.0 Request processed in 2676us (2891us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074831.264597:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1215 00000100:00100000:0.0:1769074831.270134:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625221504:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1769074831.777672:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074832.798342:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074833.309056:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625222016:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309135:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309197:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625222528:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309224:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309244:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625223040:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309289:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309336:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1769074833.309342:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625221632:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309369:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309408:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625222144:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309430:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309468:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625222656:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309490:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309504:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625223168:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309523:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309549:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625221760:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309574:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309592:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d101180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625222272:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309613:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309626:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625222784:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309661:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309682:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074833.309688:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625221888:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309714:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309730:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625222400:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309750:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309759:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625222912:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.309776:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074833.309834:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222528 00000100:00100000:2.0:1769074833.309843:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222528 00000100:00100000:2.0:1769074833.309891:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222656 00000100:00100000:2.0:1769074833.309895:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222656 00000100:00100000:2.0:1769074833.309909:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223168 00000100:00100000:2.0:1769074833.309911:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223168 00000100:00100000:2.0:1769074833.309921:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222272 00000100:00100000:2.0:1769074833.309923:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222272 00000100:00100000:2.0:1769074833.309932:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221888 00000100:00100000:2.0:1769074833.309934:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221888 00000100:00100000:2.0:1769074833.309944:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222400 00000100:00100000:2.0:1769074833.309946:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222400 00000100:00100000:2.0:1769074833.309961:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2416 00000100:00100000:2.0:1769074833.309969:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625222528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310024:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625222528:12345-0@lo:400:kworker.0 Request processed in 55us (798us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310031:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2416 00000100:00100000:2.0:1769074833.310049:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2417 00000100:00100000:2.0:1769074833.310055:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625222656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310082:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625222656:12345-0@lo:400:kworker.0 Request processed in 27us (591us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310088:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2417 00000100:00100000:2.0:1769074833.310098:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2418 00000100:00100000:2.0:1769074833.310104:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625223168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310129:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625223168:12345-0@lo:400:kworker.0 Request processed in 25us (605us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310134:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2418 00000100:00100000:2.0:1769074833.310144:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2419 00000100:00100000:2.0:1769074833.310150:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625222272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310179:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625222272:12345-0@lo:400:kworker.0 Request processed in 29us (564us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310184:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2419 00000100:00100000:2.0:1769074833.310197:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2420 00000100:00100000:2.0:1769074833.310202:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625221888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310228:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625221888:12345-0@lo:400:kworker.0 Request processed in 26us (513us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310233:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2420 00000100:00100000:2.0:1769074833.310244:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2421 00000100:00100000:2.0:1769074833.310249:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625222400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310272:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625222400:12345-0@lo:400:kworker.0 Request processed in 23us (522us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310277:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2421 00000100:00100000:2.0:1769074833.310306:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222016 00000100:00100000:2.0:1769074833.310310:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222016 00000100:00100000:2.0:1769074833.310335:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223040 00000100:00100000:2.0:1769074833.310337:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223040 00000100:00100000:2.0:1769074833.310354:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222144 00000100:00100000:2.0:1769074833.310356:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222144 00000100:00100000:2.0:1769074833.310372:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222912 00000100:00100000:2.0:1769074833.310374:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222912 00000100:00100000:2.0:1769074833.310418:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1519 00000100:00100000:2.0:1769074833.310425:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625222016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310455:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625222016:12345-0@lo:400:kworker.0 Request processed in 32us (1321us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310460:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1519 00000100:00100000:2.0:1769074833.310471:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1520 00000100:00100000:2.0:1769074833.310477:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625223040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310501:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625223040:12345-0@lo:400:kworker.0 Request processed in 25us (1212us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310506:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1520 00000100:00100000:2.0:1769074833.310517:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1521 00000100:00100000:2.0:1769074833.310522:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625222144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310546:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625222144:12345-0@lo:400:kworker.0 Request processed in 24us (1115us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310551:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1521 00000100:00100000:2.0:1769074833.310561:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1522 00000100:00100000:2.0:1769074833.310566:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625222912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310590:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625222912:12345-0@lo:400:kworker.0 Request processed in 24us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310595:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1522 00000100:00100000:2.0:1769074833.310628:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221632 00000100:00100000:2.0:1769074833.310632:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221632 00000100:00100000:2.0:1769074833.310656:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1769074833.310663:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649912850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625221632:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310692:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649912850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625221632:12345-0@lo:400:kworker.0 Request processed in 29us (1321us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310697:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1769074833.310729:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625221760 00000100:00100000:2.0:1769074833.310734:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625221760 00000100:00100000:2.0:1769074833.310760:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625222784 00000100:00100000:2.0:1769074833.310763:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625222784 00000100:00100000:2.0:1769074833.310778:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2701 00000100:00100000:2.0:1769074833.310784:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625221760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310848:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625221760:12345-0@lo:400:kworker.0 Request processed in 64us (1273us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310854:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2701 00000100:00100000:2.0:1769074833.310866:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2702 00000100:00100000:2.0:1769074833.310871:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625222784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.310897:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625222784:12345-0@lo:400:kworker.0 Request processed in 27us (1236us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074833.310903:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2702 00000100:00080000:2.0:1769074833.310931:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074833.310935:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074833.310937:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074833.310940:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074833.311006:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625221888:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311017:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625222400:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311028:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625222912:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311037:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625221760:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311046:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d101180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625222272:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311055:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625222784:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311095:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625221632:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311105:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625222144:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311115:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625222656:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311125:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625223168:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311135:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625222016:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311143:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625222528:0@lo:400:kworker.0 00000100:00100000:2.0:1769074833.311152:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625223040:0@lo:400:kworker.0 00010000:00010000:3.0:1769074833.824015:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074834.383138:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074834.383179:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074834.383212:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074834.383238:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074834.383264:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074834.383318:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617002624 00000100:00100000:3.0:1769074834.383329:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617002624 00000100:00100000:3.0:1769074834.383398:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617002752 00000100:00100000:3.0:1769074834.383402:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617002752 00000100:00100000:3.0:1769074834.383430:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2422 00000100:00100000:3.0:1769074834.383440:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007617002624:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074834.383518:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007617002624:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 78us (391us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074834.383525:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2422 00000100:00100000:3.0:1769074834.383543:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2423 00000100:00100000:3.0:1769074834.383550:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007617002752:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074834.383577:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:2408:x1855007617002752:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 28us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074834.383583:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2423 00000100:00100000:3.0:1769074834.383603:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617002880 00000100:00100000:3.0:1769074834.383606:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617002880 00000100:00100000:3.0:1769074834.383632:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003008 00000100:00100000:3.0:1769074834.383635:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003008 00000100:00100000:3.0:1769074834.383655:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1523 00000100:00100000:3.0:1769074834.383661:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617002880:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074834.383688:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617002880:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074834.383694:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1523 00000100:00100000:3.0:1769074834.383704:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1524 00000100:00100000:3.0:1769074834.383711:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617003008:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074834.383734:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617003008:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (470us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074834.383740:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1524 00000100:00100000:3.0:1769074834.383758:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617002496 00000100:00100000:3.0:1769074834.383762:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617002496 00000100:00100000:3.0:1769074834.383786:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 367 00000100:00100000:3.0:1769074834.383791:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649910450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617002496:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074834.383840:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649910450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617002496:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 48us (628us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074834.383847:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 367 00000100:00080000:3.0:1769074834.383882:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074834.383886:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074834.383888:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074834.383890:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1769074834.848302:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074835.356920:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625223296:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074835.357002:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074835.357061:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223296 00000100:00100000:2.0:1769074835.357068:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223296 00000100:00100000:2.0:1769074835.357153:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2703 00000100:00100000:2.0:1769074835.357162:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625223296:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074835.357198:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445404672 avail 1299165184 00000100:00100000:2.0:1769074835.357246:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625223296:12345-0@lo:41:osp-pre-0-1.0 Request processed in 85us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074835.357266:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2703 00000100:00100000:2.0:1769074835.357316:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625223296:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074835.361023:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625223424:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074835.361094:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074835.361181:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223424 00000100:00100000:3.0:1769074835.361189:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223424 00000100:00100000:3.0:1769074835.361236:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2704 00000100:00100000:3.0:1769074835.361244:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625223424:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074835.361289:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446150144 avail 1299910656 00000100:00100000:3.0:1769074835.361330:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625223424:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074835.361337:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2704 00000100:00100000:0.0:1769074835.361419:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625223424:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1769074835.872319:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841288 is committed 00000001:00080000:3.0:1769074835.872332:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841287 is committed 00000001:00080000:3.0:1769074835.872538:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874612 is committed 00000001:00080000:3.0:1769074835.872555:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874611 is committed 00000100:00100000:0.0:1769074835.874501:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625223552:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074835.874583:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074835.874900:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223552 00000100:00100000:3.0:1769074835.874910:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223552 00000100:00100000:3.0:1769074835.874976:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1216 00000100:00100000:3.0:1769074835.874986:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625223552:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074835.875018:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074835.875080:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625223552:12345-0@lo:13:osp-pre-0-1.0 Request processed in 83us (484us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074835.875088:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1216 00000100:00080000:0.0:1769074835.876359:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074835.876420:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625223552:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1769074835.891608:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074835.891628:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625223680:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074835.891697:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074835.891753:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223680 00000100:00100000:2.0:1769074835.891761:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223680 00000100:00100000:2.0:1769074835.891835:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1217 00000100:00100000:2.0:1769074835.891844:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625223680:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074835.891877:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 8697152 pending 0 free 3917004800 avail 3685482496 00000100:00100000:2.0:1769074835.891924:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625223680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074835.891931:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1217 00000100:00080000:2.0:1769074835.891953:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074835.892034:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874612 gen 2 00000004:00080000:2.0:1769074835.892040:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759ad80, transno 17179874610 00000004:00080000:2.0:1769074835.892051:0:3652:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 17179874612 committed transno 17179874612 00000100:00100000:2.0:1769074835.892112:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625223680:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074835.892481:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625223808:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074835.892513:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074835.892543:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223808 00000100:00100000:2.0:1769074835.892548:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223808 00000100:00100000:2.0:1769074835.892580:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1218 00000100:00100000:2.0:1769074835.892601:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625223808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074835.892613:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074835.892646:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625223808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074835.892653:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1218 00000100:00080000:2.0:1769074835.892668:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074835.892715:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841288 gen 2 00000004:00080000:2.0:1769074835.892720:0:3649:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 21474841288 committed transno 21474841288 00000100:00100000:2.0:1769074835.892752:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625223808:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1769074835.900160:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944093 is committed 00000020:00080000:2.0:1769074835.900183:0:32377:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff89b676093f00 refcount 2 committed 0 result 0 batchid 4294969365 00000020:00080000:2.0:1769074835.900188:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67a187600 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b5499cb180 00000020:00080000:2.0:1769074835.900192:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67a187e00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b676093e40 00000020:00080000:2.0:1769074835.900196:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67a187c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff89b54805f600 00000020:00080000:2.0:1769074835.900198:0:32377:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].83 00010000:00010000:2.0:1769074835.900234:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900245:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900264:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900270:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074835.900275:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900286:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900304:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x204:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074835.900307:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074835.900312:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900318:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd54fbd lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x204:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900340:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900347:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900351:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900356:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074835.900358:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900363:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900370:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x202:0x0].0x0 (ffff89b681690b00) 00010000:00010000:2.0:1769074835.900372:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074835.900375:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074835.900380:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd54fd9 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x202:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000020:00080000:3.0:1769074835.902930:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294969365 committed batchid 4294969364 00000020:00080000:3.0:1769074835.903011:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969364 00000100:00100000:2.0:1769074836.381329:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625223936:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074836.381401:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074836.382996:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625223936 00000100:00100000:0.0:1769074836.383009:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625223936 00000100:00100000:0.0:1769074836.383092:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1219 00000100:00100000:0.0:1769074836.383103:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625223936:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074836.383129:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 8697152 pending 0 free 3917004800 avail 3685482496 00000100:00100000:0.0:1769074836.383216:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625223936:12345-0@lo:13:osp-pre-1-1.0 Request processed in 116us (1814us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074836.383223:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1219 00000100:00100000:0.0:1769074836.383291:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625223936:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1769074836.383391:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074836.893534:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074837.917130:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1769074838.291636:0:37606:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 204a: FLR write/stale/resync tests with self-extending mirror ========================================================== 04:40:36 (1769074836) 00000100:00100000:0.0:1769074838.434306:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625224064:0@lo:400:kworker.0 00000100:00100000:0.0:1769074838.434382:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074838.434507:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224064 00000100:00100000:3.0:1769074838.434516:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224064 00000100:00100000:3.0:1769074838.434599:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1769074838.434609:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649912c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625224064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.435351:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625224192:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.435441:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.435468:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625224704:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.435494:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.435511:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625225216:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.435536:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.435559:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1769074838.435692:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625224320:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.435723:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.435747:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625224832:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.435769:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.435787:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625225344:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.435845:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.435866:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074838.435872:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625224448:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.435897:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.435915:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625224960:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.435937:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.435954:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625225472:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.435973:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.436016:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224192 00000100:00100000:2.0:1769074838.436023:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224192 00000100:00100000:2.0:1769074838.436058:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225216 00000100:00100000:2.0:1769074838.436061:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225216 00000100:00100000:2.0:1769074838.436081:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2705 00000100:00100000:2.0:1769074838.436089:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625224192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436133:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625224192:12345-0@lo:400:kworker.0 Request processed in 45us (691us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.436144:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2705 00000100:00100000:2.0:1769074838.436160:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2706 00000100:00100000:2.0:1769074838.436166:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efeb450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625225216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436194:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efeb450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625225216:12345-0@lo:400:kworker.0 Request processed in 28us (657us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.436199:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2706 00000100:00100000:2.0:1769074838.436612:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224704 00000100:00100000:2.0:1769074838.436617:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224704 00000100:00100000:2.0:1769074838.436649:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224320 00000100:00100000:2.0:1769074838.436652:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224320 00000100:00100000:2.0:1769074838.436663:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224832 00000100:00100000:2.0:1769074838.436665:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224832 00000100:00100000:2.0:1769074838.436677:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224960 00000100:00100000:2.0:1769074838.436679:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224960 00000100:00100000:2.0:1769074838.436694:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2424 00000100:00100000:2.0:1769074838.436701:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625224704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436739:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625224704:12345-0@lo:400:kworker.0 Request processed in 38us (1244us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.436745:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2424 00000100:00100000:2.0:1769074838.436759:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2425 00000100:00100000:2.0:1769074838.436765:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625224320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436796:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625224320:12345-0@lo:400:kworker.0 Request processed in 32us (1073us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.436802:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2425 00000100:00100000:2.0:1769074838.436835:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2426 00000100:00100000:2.0:1769074838.436842:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625224832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436872:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625224832:12345-0@lo:400:kworker.0 Request processed in 32us (1103us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.436877:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2426 00000100:00100000:2.0:1769074838.436889:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2427 00000100:00100000:2.0:1769074838.436895:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625224960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436919:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625224960:12345-0@lo:400:kworker.0 Request processed in 25us (981us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.436924:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2427 00000100:00080000:2.0:1769074838.436949:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074838.436953:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074838.436985:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625224960:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.436997:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625224320:0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.437006:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625224832:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.437330:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625224576:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.437382:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.437419:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625225088:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.437442:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.437456:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625225600:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.437476:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.437499:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625224192:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.437511:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625224704:0@lo:400:kworker.0 00000100:00100000:1.0:1769074838.437521:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625225216:0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.438854:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649912c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625224064:12345-0@lo:400:kworker.0 Request processed in 4241us (4465us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.438866:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1769074838.438945:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225088 00000100:00100000:3.0:1769074838.438953:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225088 00000100:00100000:3.0:1769074838.438994:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225600 00000100:00100000:3.0:1769074838.439104:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225600 00000100:00100000:3.0:1769074838.439127:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2428 00000100:00100000:3.0:1769074838.439134:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625225088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.439195:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625225088:12345-0@lo:400:kworker.0 Request processed in 62us (1752us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.439202:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2428 00000100:00100000:3.0:1769074838.439229:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2429 00000100:00100000:3.0:1769074838.439235:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625225600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.439264:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625225600:12345-0@lo:400:kworker.0 Request processed in 30us (1787us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.439270:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2429 00000100:00100000:3.0:1769074838.439288:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225344 00000100:00100000:3.0:1769074838.439292:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225344 00000100:00100000:3.0:1769074838.439335:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224448 00000100:00100000:3.0:1769074838.439339:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224448 00000100:00100000:3.0:1769074838.439358:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225472 00000100:00100000:3.0:1769074838.439361:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225472 00000100:00100000:3.0:1769074838.439382:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625224576 00000100:00100000:3.0:1769074838.439385:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625224576 00000100:00100000:3.0:1769074838.439406:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1525 00000100:00100000:3.0:1769074838.439412:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625225344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.439451:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625225344:12345-0@lo:400:kworker.0 Request processed in 38us (3604us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.439457:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1525 00000100:00100000:3.0:1769074838.439469:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1526 00000100:00100000:3.0:1769074838.439475:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625224448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.439501:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625224448:12345-0@lo:400:kworker.0 Request processed in 26us (3604us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.439507:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1526 00000100:00100000:3.0:1769074838.439516:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1527 00000100:00100000:3.0:1769074838.439522:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625225472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.439549:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625225472:12345-0@lo:400:kworker.0 Request processed in 27us (3575us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.439558:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1527 00000100:00100000:2.0:1769074838.440166:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1528 00000100:00100000:2.0:1769074838.440177:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625224576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074838.440236:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625224576:12345-0@lo:400:kworker.0 Request processed in 62us (2852us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.440241:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1528 00000100:00100000:3.0:1769074838.441333:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074838.441393:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625224576:0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.441408:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625225088:0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.441420:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625225600:0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.441478:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003136 00000100:00100000:3.0:1769074838.441483:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003136 00000100:00100000:3.0:1769074838.441524:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13494 00000100:00100000:3.0:1769074838.441532:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66075:x1855007617003136:12345-192.168.204.60@tcp:41:stat.0 00000004:00000024:3.0:1769074838.441694:0:6512:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7831998464 avail 7368953856 00000100:00100000:3.0:1769074838.441759:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66075:x1855007617003136:12345-192.168.204.60@tcp:41:stat.0 Request processed in 226us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.441767:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13494 00000100:00100000:0.0:1769074838.445645:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625224064:0@lo:400:kworker.0 00000100:00080000:0.0:1769074838.445834:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074838.445841:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074838.445844:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074838.445847:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074838.445893:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625224448:0@lo:400:kworker.0 00000100:00100000:0.0:1769074838.445907:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625225472:0@lo:400:kworker.0 00000100:00100000:0.0:1769074838.445920:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625225344:0@lo:400:kworker.0 00000100:00100000:3.0:1769074838.508763:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.508903:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003264 00000100:00100000:0.0:1769074838.508914:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003264 00000100:00100000:0.0:1769074838.508964:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13495 00000100:00100000:0.0:1769074838.508972:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66083:x1855007617003264:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.508990:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.509010:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c664 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.509080:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509095:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509108:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509123:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509129:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509135:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509176:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509182:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509189:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509196:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509202:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.509219:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509229:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509376:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074838.509379:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.509384:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509390:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd550c0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509403:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509408:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509413:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509417:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.509420:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509425:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509431:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x84e5157e (ffff89b643a80100) 00010000:00010000:0.0:1769074838.509433:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.509436:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509441:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd550c7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074838.509469:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550b9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c664 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.509478:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074838.509480:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.509483:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165400/0x7fe2323adfd550b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c664 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.509491:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165400, rc 0) 00000100:00100000:0.0:1769074838.509554:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66083:x1855007617003264:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 582us (802us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074838.509658:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13495 00000100:00100000:0.0:1769074838.527162:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074838.527367:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003520 00000100:00100000:3.0:1769074838.527377:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003520 00000100:00100000:3.0:1769074838.527436:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13496 00000100:00100000:3.0:1769074838.527446:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66083:x1855007617003520:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074838.527501:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074838.527527:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164000/0x7fe2323adfd5500a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c641 expref: 10 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074838.527544:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074838.527549:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074838.527610:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.527623:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.527632:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.527644:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x84e5157e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.527650:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.527656:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000040:00080000:3.0:1769074838.529471:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].84, 2168 off88472 00000040:00080000:3.0:1769074838.529654:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000405:0x2:0x0].6, 2368 off44992 00000004:00080000:3.0:1769074838.529663:0:6512:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b5492660c0 with version 17 00000004:00080000:0.0:1769074838.530746:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 17 rpc_version 17 00000020:00100000:1.0:1769074838.530965:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1769074838.530977:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b5444a4000 x1855007625225856/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1769074838.531008:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625225856:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074838.531135:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074838.531214:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604e40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074838.533225:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225856 00000100:00100000:3.0:1769074838.533240:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225856 00000100:00100000:3.0:1769074838.533313:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2707 00000100:00100000:3.0:1769074838.533323:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625225856:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1769074838.534224:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625225856:12345-0@lo:1000:osp_up1-0.0 Request processed in 900us (3087us total) trans 4294968350 rc 0/0 00000100:00100000:3.0:1769074838.534233:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2707 00000100:00100000:1.0:1769074838.534372:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625225856:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1769074838.534728:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,1 mode: --/PW res: [0x200000405:0x205:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.534744:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.534759:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.534777:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.534784:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.534789:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.536731:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 3/0,1 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.536747:0:6512:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 3/0,1 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.536758:0:6512:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 3/0,1 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6512 initiator: MDT0 00000100:00100000:3.0:1769074838.536785:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6512:1855007625225984:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1769074838.536861:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074838.536896:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266840 going to sleep for 11 seconds 00000100:00100000:1.0:1769074838.537109:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625225984 00000100:00100000:1.0:1769074838.537118:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625225984 00000100:00100000:1.0:1769074838.537224:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2708 00000100:00100000:1.0:1769074838.537233:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6512:x1855007625225984:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1769074838.537253:0:35531:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074838.537294:0:35531:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 2/0,0 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7fe2323adfd550ea expref: -99 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.537327:0:35531:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7fe2323adfd550ea expref: 7 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.537342:0:35531:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd550ea expref: 7 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.537350:0:35531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697c00) 00010000:00010000:1.0:1769074838.537353:0:35531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.537356:0:35531:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b641520000, rc 0) 00000100:00100000:1.0:1769074838.537405:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6512:x1855007625225984:12345-0@lo:101:mdt00_000.0 Request processed in 173us (544us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074838.537412:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2708 00000100:00100000:3.0:1769074838.537477:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6512:1855007625225984:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1769074838.537491:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074838.537500:0:6512:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1769074838.537521:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000007f3aed7b reply state 000000004d74d318 transno 8589944094 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537533:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537539:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x7:0x0].0x0 (ffff89b681690600) 00010000:00010000:3.0:1769074838.537541:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074838.537545:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1769074838.537562:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074838.537570:0:6512:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00000004:00080000:3.0:1769074838.537581:0:6512:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000405:0x7:0x0] 00000004:00010000:3.0:1769074838.537588:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000007f3aed7b reply state 000000004d74d318 transno 8589944094 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537595:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537600:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x205:0x0].0x0 (ffff89b681690e00) 00010000:00010000:3.0:1769074838.537603:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074838.537604:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1769074838.537609:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x205:0x0]: returning size 4096 00010000:00010000:3.0:1769074838.537620:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537625:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537631:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537637:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537644:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074838.537648:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537655:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537664:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074838.537667:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074838.537669:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537674:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd550ce lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000004:00010000:3.0:1769074838.537687:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000007f3aed7b reply state 000000004d74d318 transno 8589944094 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537694:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.537699:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x84e5157e (ffff89b681690f00) 00010000:00010000:3.0:1769074838.537701:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074838.537703:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 2 00000100:00100000:3.0:1769074838.537790:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66083:x1855007617003520:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 10344us (10636us total) trans 8589944094 rc 0/0 00000100:00100000:3.0:1769074838.537797:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13496 00000100:00100000:3.0:1769074838.585284:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.589342:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003648 00000100:00100000:0.0:1769074838.589360:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003648 00000100:00100000:2.0:1769074838.589554:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13497 00000100:00100000:2.0:1769074838.589567:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66089:x1855007617003648:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074838.589592:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074838.589614:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd550f8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c66b expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.589685:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd550ff lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.589699:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd550ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.589708:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd550ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074838.589748:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074838.589751:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074838.589765:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd550ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.589782:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd550f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c66b expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.589796:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd550ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c66b expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.589835:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074838.589837:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074838.589840:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074838.589905:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:66089:x1855007617003648:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 342us (4637us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.589910:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13497 00000100:00100000:0.0:1769074838.596050:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074838.596225:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003776 00000100:00100000:2.0:1769074838.596244:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003776 00000100:00100000:2.0:1769074838.596305:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13498 00000100:00100000:2.0:1769074838.596316:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:66089:x1855007617003776:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074838.596338:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074838.596357:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55106 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c672 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.596424:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596438:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596449:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596459:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596466:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596472:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596548:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5511b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x205:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596556:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5511b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596575:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5511b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074838.596660:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x205:0x0]: returning size 4096 00000004:00200000:2.0:1769074838.596666:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1769074838.596683:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5511b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596695:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596701:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596707:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596713:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596719:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.596722:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596730:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596742:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074838.596745:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074838.596749:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596754:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5510d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596765:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596771:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596776:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596781:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.596783:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596789:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596796:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x84e5157e (ffff89b681690f00) 00010000:00010000:2.0:1769074838.596798:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074838.596801:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.596859:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55114 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.597054:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55106 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c672 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.597070:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88a00/0x7fe2323adfd5511b lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c672 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.597077:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x205:0x0].0x0 (ffff89b681690e00) 00010000:00010000:2.0:1769074838.597080:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1769074838.597083:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88a00, rc 0) 00000100:00100000:2.0:1769074838.597141:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:66089:x1855007617003776:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 826us (1100us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.597149:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13498 00000100:00100000:1.0:1769074838.600747:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.603768:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617003904 00000100:00100000:0.0:1769074838.603784:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617003904 00000100:00100000:0.0:1769074838.603871:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13499 00000100:00100000:0.0:1769074838.603882:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:66089:x1855007617003904:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.603901:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.603921:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd55122 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c679 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.603981:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55129 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.603998:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55129 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.604010:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55129 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074838.604058:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd0:0x7:0x0]: returning size 4096 00000004:00200000:0.0:1769074838.604063:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1769074838.604097:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164800/0x7fe2323adfd55122 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c679 expref: 15 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.604115:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682165600/0x7fe2323adfd55129 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c679 expref: 14 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.604123:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x7:0x0].0x0 (ffff89b681690600) 00010000:00010000:0.0:1769074838.604126:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1769074838.604130:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165600, rc 0) 00000100:00100000:0.0:1769074838.604215:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66089:x1855007617003904:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 324us (3467us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074838.604222:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13499 00000100:00100000:1.0:1769074838.609104:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.609199:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617004032 00000100:00100000:0.0:1769074838.609218:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617004032 00000100:00100000:0.0:1769074838.609390:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13500 00000100:00100000:0.0:1769074838.609400:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:66089:x1855007617004032:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.609420:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.609721:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165200/0x7fe2323adfd55130 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c680 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.609778:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.609789:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 2/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd550ea expref: 6 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.609795:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 4/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.609839:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7fe2323adfd550ea expref: 6 pid: 35531 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074838.609850:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10283:1855007625226112:0@lo:104: 00000100:00100000:0.0:1769074838.609891:0:10283:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074838.609936:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b545b03180 going to sleep for 11 seconds 00000100:00080000:2.0:1769074838.610159:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074838.610218:0:6497:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625226112 00000100:00100000:1.0:1769074838.610224:0:6497:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625226112 00000100:00100000:1.0:1769074838.610357:0:6497:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:1.0:1769074838.610362:0:6497:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10283:x1855007625226112:12345-0@lo:104: 00010000:00010000:1.0:1769074838.610370:0:6497:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.610416:0:6497:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1769074838.610421:0:6497:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10283:x1855007625226112:12345-0@lo:104: Request processed in 60us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074838.610432:0:6497:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00010000:00010000:2.0:1769074838.610452:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074838.610461:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.610470:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc1908290) 00010000:00010000:2.0:1769074838.610476:0:6502:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1769074838.610485:0:6502:0:(mdt_handler.c:3944:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.610514:0:6502:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 2/0,0 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.610520:0:6502:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1769074838.610544:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10283:1855007625226112:0@lo:104: 00010000:00010000:2.0:1769074838.610693:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 1/0,0 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.610701:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd550ea lrc: 0/0,0 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd550f1 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.610719:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074838.610720:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074838.610722:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074838.610754:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625226240:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1769074838.610783:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.610840:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625226240 00000100:00100000:2.0:1769074838.610846:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625226240 00010000:00010000:2.0:1769074838.610868:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7fe2323adfd550ea expref: 7 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.610889:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074838.610895:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7205 00000100:00100000:2.0:1769074838.610902:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625226240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1769074838.610912:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1769074838.613050:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968350 is committed 00010000:00010000:2.0:1769074838.613164:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd550f1 lrc: 0/0,0 mode: --/PW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7fe2323adfd550ea expref: 6 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.613187:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697c00) 00010000:00010000:2.0:1769074838.613192:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.613211:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.613219:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 4/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.613429:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074838.613517:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625226240:12345-0@lo:103:ldlm_bl.0 Request processed in 2614us (2730us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.613524:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7205 00010000:00010000:0.0:1769074838.613591:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.613599:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:0.0:1769074838.613650:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000405:0x7:0x0]: returning size 4096 00000004:00200000:0.0:1769074838.613654:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000100:00100000:3.0:1769074838.613766:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625226240:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1769074838.613998:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165200/0x7fe2323adfd55130 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c680 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.614014:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c680 expref: 6 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.614027:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697c00) 00010000:00010000:0.0:1769074838.614031:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1769074838.614033:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165a00, rc 0) 00000100:00100000:0.0:1769074838.614093:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66089:x1855007617004032:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4693us (4997us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074838.614100:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13500 00000100:00100000:3.0:1769074838.616390:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.616465:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617004160 00000100:00100000:0.0:1769074838.616473:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617004160 00000100:00100000:0.0:1769074838.616519:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13501 00000100:00100000:0.0:1769074838.616526:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66089:x1855007617004160:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.616543:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.616574:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165400/0x7fe2323adfd5513e lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c687 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.616636:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.616647:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.616655:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.616977:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.616984:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.616989:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617041:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617048:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617053:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617059:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617065:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.617069:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617076:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617088:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697c00) 00010000:00010000:0.0:1769074838.617091:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.617096:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617101:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164000/0x7fe2323adfd55145 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617114:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617119:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617125:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617130:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682165000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.617133:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617139:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617145:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317800) 00010000:00010000:0.0:1769074838.617147:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.617150:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617155:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165000/0x7fe2323adfd5514c lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.617188:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165400/0x7fe2323adfd5513e lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c687 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.617198:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697c00) 00010000:00010000:0.0:1769074838.617200:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.617453:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165400/0x7fe2323adfd5513e lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c687 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.617466:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682165400, rc 0) 00000100:00100000:0.0:1769074838.617720:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66089:x1855007617004160:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1136us (1280us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074838.617726:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13501 00000100:00100000:1.0:1769074838.620418:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.620545:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617004288 00000100:00100000:0.0:1769074838.620718:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617004288 00000100:00100000:0.0:1769074838.620797:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13502 00000100:00100000:0.0:1769074838.620825:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66089:x1855007617004288:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.620842:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.620858:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd55153 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c68e expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.620927:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd5515a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x205:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.620941:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd5515a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.620953:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd5515a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074838.621038:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x205:0x0]: returning size 4096 00000004:00200000:0.0:1769074838.621043:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1769074838.621062:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd5515a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.621086:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164600/0x7fe2323adfd55153 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c68e expref: 16 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.621104:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b682164400/0x7fe2323adfd5515a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x205:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c68e expref: 15 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.621113:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x205:0x0].0x0 (ffff89b681690e00) 00010000:00010000:0.0:1769074838.621116:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769074838.621119:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b682164400, rc 0) 00000100:00100000:0.0:1769074838.621248:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:66089:x1855007617004288:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 443us (839us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074838.621259:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13502 00000100:00100000:1.0:1769074838.626538:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.626872:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617004544 00000100:00100000:0.0:1769074838.626883:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617004544 00000100:00100000:0.0:1769074838.626935:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13503 00000100:00100000:0.0:1769074838.626945:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66089:x1855007617004544:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.626965:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.626968:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1769074838.626997:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165a00/0x7fe2323adfd55137 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c680 expref: 6 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.627015:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697c00) 00010000:00010000:0.0:1769074838.627025:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074838.627041:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165200/0x7fe2323adfd55161 lrc: 2/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c695 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.627143:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 3/0,1 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.627158:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 3/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.627168:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 3/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.627180:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,1 mode: --/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.627185:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.627191:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.628518:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.628537:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.628546:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:0.0:1769074838.628658:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 0 00010000:00010000:0.0:1769074838.628772:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 4/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629058:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 4/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629063:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 3/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629070:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 4/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629076:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.629080:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 4/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629087:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 5/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629100:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317c00) 00010000:00010000:0.0:1769074838.629103:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.629107:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 3/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629116:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164800/0x7fe2323adfd55168 lrc: 0/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000004:00010000:0.0:1769074838.629129:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000007256a136 reply state 0000000036d78ce0 transno 4294968351 ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629138:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.629144:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317200) 00010000:00010000:0.0:1769074838.629146:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1769074838.629148:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1769074838.629181:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682165200/0x7fe2323adfd55161 lrc: 0/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.629200:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.629216:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:0.0:1769074838.629218:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074838.629220:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135000, rc 0) 00000100:00100000:0.0:1769074838.629383:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66089:x1855007617004544:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 2438us (2852us total) trans 4294968351 rc 0/0 00000100:00100000:0.0:1769074838.629392:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13503 00000100:00100000:3.0:1769074838.634556:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.645044:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617004800 00000100:00100000:0.0:1769074838.645061:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617004800 00000100:00100000:0.0:1769074838.645124:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13504 00000100:00100000:0.0:1769074838.645134:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66089:x1855007617004800:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.645158:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.645177:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd55184 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c69c expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.645468:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074838.645484:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.645493:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 4/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.645529:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.645542:0:10283:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 9 pid: 10283 timeout: 1926 lvb_type: 0 00000100:00100000:0.0:1769074838.645884:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10283:1855007625226368:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074838.645982:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6480f3300 going to sleep for 11 seconds 00000100:00100000:3.0:1769074838.653273:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.653493:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10283:1855007625226368:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074838.653521:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074838.654261:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617004928 00000100:00100000:2.0:1769074838.654272:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617004928 00010000:00010000:2.0:1769074838.654315:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 10 pid: 10283 timeout: 1926 lvb_type: 0 00000100:00100000:2.0:1769074838.654403:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7206 00000100:00100000:2.0:1769074838.654415:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617004928:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074838.654443:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074838.654452:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 10 pid: 10283 timeout: 1926 lvb_type: 0 00010000:00010000:2.0:1769074838.654465:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 10 pid: 10283 timeout: 1926 lvb_type: 0 00010000:00010000:2.0:1769074838.654495:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55176 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c695 expref: 10 pid: 10283 timeout: 1926 lvb_type: 0 00010000:00010000:2.0:1769074838.654511:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:2.0:1769074838.654516:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.654525:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.654533:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 4/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.654572:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074838.654642:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617004928:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 229us (1380us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.654650:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7206 00010000:00010000:0.0:1769074838.654712:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.654720:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:0.0:1769074838.654743:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 0 00000004:00080000:0.0:1769074838.654941:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x35:0x0], index 0: 0 00000004:00080000:0.0:1769074838.654949:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x35:0x0], index 1: 0 00010000:00010000:0.0:1769074838.655049:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655058:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 2/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655064:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655070:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.655073:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655081:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 4/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655091:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:0.0:1769074838.655094:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074838.655097:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 2/0,0 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655103:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd5518b lrc: 0/0,0 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074838.655144:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd55184 lrc: 1/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c69c expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.655154:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:0.0:1769074838.655156:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074838.655160:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd55184 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c69c expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074838.655178:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134000, rc 0) 00000100:00100000:0.0:1769074838.655228:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66089:x1855007617004800:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 10096us (20684us total) trans 4294968352 rc 301/301 00000100:00100000:0.0:1769074838.655235:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13504 00000100:00100000:3.0:1769074838.658360:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.663292:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617005184 00000100:00100000:0.0:1769074838.663310:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617005184 00000100:00100000:0.0:1769074838.663593:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4730 00000100:00100000:0.0:1769074838.663605:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66089:x1855007617005184:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074838.663934:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66089:x1855007617005184:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 331us (5582us total) trans 4294968353 rc 0/0 00000100:00100000:0.0:1769074838.663943:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4730 00000100:00100000:1.0:1769074838.666521:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.666950:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617005440 00000100:00100000:0.0:1769074838.666961:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617005440 00000100:00100000:0.0:1769074838.667008:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4731 00000100:00100000:0.0:1769074838.667018:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66089:x1855007617005440:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074838.667327:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66089:x1855007617005440:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 309us (814us total) trans 4294968354 rc 0/0 00000100:00100000:0.0:1769074838.667336:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4731 00000100:00100000:3.0:1769074838.674470:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.675256:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617005696 00000100:00100000:0.0:1769074838.675267:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617005696 00000100:00100000:1.0:1769074838.675462:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13505 00000100:00100000:1.0:1769074838.675472:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66090:x1855007617005696:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074838.675496:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074838.675521:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b641521e00/0x7fe2323adfd55199 lrc: 2/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6a3 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.675687:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074838.675702:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675713:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675723:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074838.675730:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675735:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675848:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29400/0x7fe2323adfd551ae lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074838.675857:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29400/0x7fe2323adfd551ae lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675864:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29400/0x7fe2323adfd551ae lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:1.0:1769074838.675878:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 0 00010000:00010000:1.0:1769074838.675959:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675966:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675972:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675978:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675984:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641520e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.675988:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.675994:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676006:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074838.676009:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.676013:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676019:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520e00/0x7fe2323adfd551a0 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676034:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676039:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676048:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676053:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b641520000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.676056:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676061:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676074:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317200) 00010000:00010000:1.0:1769074838.676077:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.676079:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676085:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641520000/0x7fe2323adfd551a7 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.676120:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b641521e00/0x7fe2323adfd55199 lrc: 0/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6a3 expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.676136:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29400/0x7fe2323adfd551ae lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6a3 expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.676144:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697400) 00010000:00010000:1.0:1769074838.676146:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074838.676148:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29400, rc 0) 00000100:00100000:1.0:1769074838.676229:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:66090:x1855007617005696:12345-192.168.204.60@tcp:101:dd.0 Request processed in 755us (1764us total) trans 4294968355 rc 0/0 00000100:00100000:1.0:1769074838.676238:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13505 00000100:00100000:3.0:1769074838.691621:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074838.692063:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617005952 00000100:00100000:1.0:1769074838.692074:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617005952 00000100:00100000:1.0:1769074838.692149:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13506 00000100:00100000:1.0:1769074838.692160:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:66090:x1855007617005952:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074838.692188:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074838.692192:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074838.692230:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29400/0x7fe2323adfd551ae lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6a3 expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.692252:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697400) 00010000:00010000:1.0:1769074838.692262:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074838.692277:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29800/0x7fe2323adfd551bc lrc: 2/0,0 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6aa expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.692331:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd551c3 lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074838.692343:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd551c3 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074838.692352:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd551c3 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00080000:1.0:1769074838.692570:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x36:0x0], index 1: 0 00000040:00080000:1.0:1769074838.693083:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].63, 88 off13208 00000004:00080000:1.0:1769074838.693097:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1826535288575ns, next at 1384406513989ns, rc = 0 00000040:00080000:1.0:1769074838.693146:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].63, 88 off13136 00000004:00080000:1.0:1769074838.693153:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1826535345614ns, next at 1394702964651ns, rc = 0 00000040:00080000:1.0:1769074838.693211:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].64, 88 off13224 00000004:00080000:1.0:1769074838.693217:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1826535410070ns, next at 1394702964651ns, rc = 0 00000001:00080000:3.0:1769074838.697531:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968356 is committed 00000004:00080000:3.0:1769074838.697552:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 62 committed 00000004:00080000:3.0:1769074838.697574:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 61 committed 00000004:00080000:3.0:1769074838.697577:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 61 committed 00010000:00010000:2.0:1769074838.697652:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00000100:00100000:0.0:1769074838.697949:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625226624:0@lo:2:osp-syn-1-1.0 00000100:00100000:0.0:1769074838.698012:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074838.698051:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074838.698057:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625226496:0@lo:2:osp-syn-1-1.0 00000100:00100000:0.0:1769074838.698082:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074838.698117:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625226624 00000100:00100000:0.0:1769074838.698124:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625226624 00000100:00100000:0.0:1769074838.698177:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625226496 00000100:00100000:0.0:1769074838.698181:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625226496 00000100:00100000:0.0:1769074838.698198:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14470 00000100:00100000:0.0:1769074838.698216:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3650:x1855007625226624:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:0.0:1769074838.698442:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625226624:12345-0@lo:2:osp-syn-1-1.0 Request processed in 225us (428us total) trans 17179874613 rc 0/0 00000100:00100000:0.0:1769074838.698449:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14470 00000100:00100000:0.0:1769074838.698465:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14471 00000100:00100000:0.0:1769074838.698470:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3650:x1855007625226496:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:0.0:1769074838.698643:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625226496:12345-0@lo:2:osp-syn-1-1.0 Request processed in 172us (559us total) trans 17179874614 rc 0/0 00000100:00100000:0.0:1769074838.698649:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14471 00000004:00080000:0.0:1769074838.698875:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6656daa00/2, rc 0, transno 17179874614 00000100:00100000:0.0:1769074838.698881:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625226496:0@lo:2:osp-syn-1-1.0 00000004:00080000:0.0:1769074838.698890:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6656d9c00/2, rc 0, transno 17179874613 00000100:00100000:0.0:1769074838.698893:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625226624:0@lo:2:osp-syn-1-1.0 00010000:00010000:2.0:1769074838.699432:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 2/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.699441:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.699447:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b682164200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.699452:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.699460:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 4/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.699477:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317200) 00010000:00010000:2.0:1769074838.699480:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074838.699484:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 2/0,0 mode: --/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.699490:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b682164200/0x7fe2323adfd5516f lrc: 0/0,0 mode: --/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:1.0:1769074838.711044:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625226752:0@lo:2:osp-syn-0-1.0 00000100:00100000:1.0:1769074838.711115:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074838.711289:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625226752 00000100:00100000:2.0:1769074838.711297:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625226752 00000100:00100000:2.0:1769074838.711355:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14472 00000100:00100000:2.0:1769074838.711365:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625226752:12345-0@lo:2:osp-syn-0-1.0 00000100:00100000:2.0:1769074838.719029:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625226752:12345-0@lo:2:osp-syn-0-1.0 Request processed in 7655us (7903us total) trans 21474841289 rc 0/0 00000100:00100000:2.0:1769074838.719044:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14472 00000004:00080000:1.0:1769074838.719324:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2aa00/2, rc 0, transno 21474841289 00000100:00100000:1.0:1769074838.719332:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625226752:0@lo:2:osp-syn-0-1.0 00010000:00010000:1.0:1769074838.719402:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd551c3 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6aa expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.719415:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074838.719418:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:1.0:1769074838.719435:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29800/0x7fe2323adfd551bc lrc: 0/0,0 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6aa expref: 7 pid: 10283 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1769074838.719453:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd551c3 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0xa/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6aa expref: 6 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.719483:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074838.719486:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:1.0:1769074838.719489:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28c00, rc 0) 00000100:00100000:1.0:1769074838.720464:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66090:x1855007617005952:12345-192.168.204.60@tcp:101:dd.0 Request processed in 28298us (28844us total) trans 4294968356 rc 0/0 00000100:00100000:1.0:1769074838.720475:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13506 00000100:00100000:3.0:1769074838.723236:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074838.723424:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617006080 00000100:00100000:2.0:1769074838.723434:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617006080 00000100:00100000:2.0:1769074838.723495:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13507 00000100:00100000:2.0:1769074838.723505:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66090:x1855007617006080:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074838.723529:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074838.723561:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd551ca lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6b1 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.723594:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88c00/0x7fe2323adfd551d1 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074838.723609:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88c00/0x7fe2323adfd551d1 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.723624:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88c00/0x7fe2323adfd551d1 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074838.723755:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd551ca lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6b1 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.723773:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88c00/0x7fe2323adfd551d1 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6b1 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.723783:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074838.723785:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1769074838.723790:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88c00, rc 0) 00000100:00100000:2.0:1769074838.723866:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66090:x1855007617006080:12345-192.168.204.60@tcp:101:dd.0 Request processed in 363us (640us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074838.723874:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13507 00000100:00100000:3.0:1769074838.729060:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074838.729441:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617006208 00000100:00100000:1.0:1769074838.729454:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617006208 00000100:00100000:1.0:1769074838.729601:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8744 00000100:00100000:1.0:1769074838.729612:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617006208:12345-192.168.204.60@tcp:4:dd.0 00002000:00010000:1.0:1769074838.729672:0:28434:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x35:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074838.729681:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.729696:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074838.729698:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074838.729703:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.729720:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00080000:2.0:1769074838.731266:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1769074838.786053:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841290 is committed 00000001:00080000:3.0:1769074838.786067:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841289 is committed 00002000:00010000:2.0:1769074838.786247:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:2.0:1769074838.786252:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb mtime from disk: 0 -> 1769074838 00002000:00010000:2.0:1769074838.786255:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb ctime from disk: 0 -> 1769074838 00002000:00010000:2.0:1769074838.786257:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb blocks from disk: 0 -> 4096 00010000:00010000:2.0:1769074838.786271:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.786283:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.786292:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.786303:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074838.786307:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.786316:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.786334:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.786341:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b543a29400/0x7fe2323adfd551d8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074838.786615:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617006208:12345-192.168.204.60@tcp:4:dd.0 Request processed in 57000us (57560us total) trans 21474841290 rc 0/0 00000100:00100000:2.0:1769074838.786625:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8744 00000100:00100000:3.0:1769074838.796485:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.796707:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617006464 00000100:00100000:0.0:1769074838.796717:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617006464 00000100:00100000:0.0:1769074838.796781:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4732 00000100:00100000:0.0:1769074838.796792:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66090:x1855007617006464:12345-192.168.204.60@tcp:35:dd.0 00000100:00100000:0.0:1769074838.797141:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66090:x1855007617006464:12345-192.168.204.60@tcp:35:dd.0 Request processed in 343us (658us total) trans 4294968357 rc 0/0 00000100:00100000:0.0:1769074838.797153:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4732 00000100:00100000:2.0:1769074838.825976:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074838.826172:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617006720 00000100:00100000:3.0:1769074838.826182:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617006720 00000100:00100000:3.0:1769074838.826246:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13508 00000100:00100000:3.0:1769074838.826255:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66091:x1855007617006720:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074838.826278:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074838.826300:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd551df lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6b8 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074838.826370:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.826384:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826395:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826405:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.826411:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826417:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826499:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826506:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826512:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826518:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826523:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074838.826527:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826534:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826550:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b6543eab00) 00010000:00010000:3.0:1769074838.826553:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074838.826557:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826563:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd551e6 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826574:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826580:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826586:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826590:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbe00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074838.826593:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826598:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826603:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b6543ea800) 00010000:00010000:3.0:1769074838.826606:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074838.826608:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826613:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd551ed lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.826639:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd551df lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6b8 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074838.826648:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b6543eab00) 00010000:00010000:3.0:1769074838.826650:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074838.826654:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd551df lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6b8 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074838.826664:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febac00, rc 0) 00000100:00100000:3.0:1769074838.826723:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66091:x1855007617006720:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 469us (756us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074838.826730:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13508 00000100:00100000:3.0:1769074838.832283:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074838.832681:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617006976 00000100:00100000:2.0:1769074838.832692:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617006976 00000100:00100000:2.0:1769074838.832747:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13509 00000100:00100000:2.0:1769074838.832756:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:66091:x1855007617006976:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074838.832777:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074838.832799:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd551f4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6bf expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1769074838.832988:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x205:0x0]: returning size 4096 00010000:00010000:2.0:1769074838.833100:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd551f4 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6bf expref: 17 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.833116:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074838.833119:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074838.833125:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd551f4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6bf expref: 17 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.833139:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074838.833215:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:66091:x1855007617006976:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 458us (941us total) trans 8589944095 rc 301/301 00000100:00100000:2.0:1769074838.833223:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13509 00000100:00100000:1.0:1769074838.839958:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074838.840049:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617007104 00000100:00100000:3.0:1769074838.840060:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617007104 00000100:00100000:3.0:1769074838.840118:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13510 00000100:00100000:3.0:1769074838.840127:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66091:x1855007617007104:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074838.840221:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.840238:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840249:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840261:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.840268:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840274:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840339:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074838.840349:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840356:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:3.0:1769074838.840398:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00000004:00200000:3.0:1769074838.840402:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074838.840414:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840428:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840433:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840772:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840781:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840787:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074838.840791:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.840798:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843316:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b6543eab00) 00010000:00010000:3.0:1769074838.843325:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074838.843334:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843346:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55202 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843374:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843381:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843387:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843393:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbe00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074838.843396:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843402:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843419:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b6543ea800) 00010000:00010000:3.0:1769074838.843422:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074838.843426:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843432:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55209 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843446:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843452:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843457:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843462:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074838.843464:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843469:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843476:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:3.0:1769074838.843478:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074838.843481:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074838.843486:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55210 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:3.0:1769074838.843601:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66091:x1855007617007104:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 3470us (3649us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074838.843612:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13510 00000100:00100000:0.0:1769074838.848066:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074838.848149:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617007360 00000100:00100000:1.0:1769074838.848489:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617007360 00000100:00100000:1.0:1769074838.848568:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4733 00000100:00100000:1.0:1769074838.848589:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:66091:x1855007617007360:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074838.848860:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:66091:x1855007617007360:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 282us (804us total) trans 8589944096 rc 0/0 00000100:00100000:1.0:1769074838.848868:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4733 00000100:00100000:1.0:1769074838.889337:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074838.889888:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617007616 00000100:00100000:3.0:1769074838.889898:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617007616 00000100:00100000:3.0:1769074838.889963:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13511 00000100:00100000:3.0:1769074838.889973:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66097:x1855007617007616:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074838.889993:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074838.892787:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55217 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6c6 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.892901:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.892916:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.892929:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.892943:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.892950:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.892957:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893054:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893062:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893069:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893075:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893081:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.893085:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893093:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893105:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:2.0:1769074838.893108:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074838.893112:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893119:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5521e lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893132:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893139:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893144:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893150:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.893152:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893158:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893165:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436a000) 00010000:00010000:2.0:1769074838.893168:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074838.893170:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893176:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55225 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.893222:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55217 lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6c6 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.893232:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:2.0:1769074838.893235:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074838.893239:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55217 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6c6 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.893250:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074838.893374:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66097:x1855007617007616:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 3401us (4045us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074838.893383:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13511 00000100:00100000:2.0:1769074838.901977:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074838.902321:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617007872 00000100:00100000:0.0:1769074838.902332:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617007872 00000100:00100000:0.0:1769074838.902396:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13512 00000100:00100000:0.0:1769074838.902406:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:66097:x1855007617007872:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074838.902635:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074838.902657:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5522c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6cd expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.906238:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29e00/0x7fe2323adfd55233 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x205:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.906262:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29e00/0x7fe2323adfd55233 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x205:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.906273:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29e00/0x7fe2323adfd55233 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x205:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:1.0:1769074838.906294:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x205:0x0]: returning size 4096 00010000:00010000:1.0:1769074838.906434:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5522c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6cd expref: 20 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.906459:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29e00/0x7fe2323adfd55233 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x205:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6cd expref: 19 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.906467:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x205:0x0].0x0 (ffff89b681690e00) 00010000:00010000:1.0:1769074838.906470:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1769074838.906473:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29e00, rc 0) 00000100:00100000:1.0:1769074838.906557:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66097:x1855007617007872:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4152us (4590us total) trans 8589944097 rc 0/0 00000100:00100000:1.0:1769074838.906566:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13512 00000100:00100000:0.0:1769074838.912492:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074838.912576:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617008000 00000100:00100000:1.0:1769074838.912587:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617008000 00000100:00100000:1.0:1769074838.912653:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13513 00000100:00100000:1.0:1769074838.912663:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66097:x1855007617008000:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:1.0:1769074838.912734:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.912750:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912761:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912773:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.912779:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912785:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912890:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.912901:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912908:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:1.0:1769074838.912949:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00000004:00200000:1.0:1769074838.912953:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074838.912964:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912977:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912984:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912989:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.912996:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913014:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.913018:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913267:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913281:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697100) 00010000:00010000:1.0:1769074838.913285:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.913289:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913295:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55241 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913312:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913318:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913322:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913327:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.913330:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913336:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913343:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b681697300) 00010000:00010000:1.0:1769074838.913346:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.913348:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913354:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55248 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913364:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913369:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913375:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913380:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.913383:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913388:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913395:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074838.913397:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.913400:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.913405:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5524f lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:1.0:1769074838.913521:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66097:x1855007617008000:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 859us (1038us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074838.913530:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13513 00010000:00010000:2.0:1769074838.940989:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074838.963425:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074838.964006:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617008256 00000100:00100000:1.0:1769074838.964017:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617008256 00000100:00100000:1.0:1769074838.964334:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13514 00000100:00100000:1.0:1769074838.964344:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66102:x1855007617008256:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074838.964365:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074838.964399:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd55256 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6d4 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.964493:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.964508:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964518:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964530:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.964537:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964542:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964620:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964627:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964633:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964638:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964644:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.964649:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964656:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964672:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697300) 00010000:00010000:1.0:1769074838.964675:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.964679:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964685:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd5525d lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964699:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964704:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964709:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964714:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074838.964717:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964723:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964729:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b681697100) 00010000:00010000:1.0:1769074838.964732:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074838.964735:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964740:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55264 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.964769:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd55256 lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6d4 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.964779:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697300) 00010000:00010000:1.0:1769074838.964782:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074838.964785:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd55256 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6d4 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074838.964796:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29200, rc 0) 00000100:00100000:1.0:1769074838.964895:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66102:x1855007617008256:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 552us (1488us total) trans 0 rc 301/301 00000100:00100000:1.0:1769074838.964904:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13514 00000100:00100000:2.0:1769074838.975539:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074838.975661:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617008384 00000100:00100000:1.0:1769074838.975672:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617008384 00000100:00100000:1.0:1769074838.975736:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13515 00000100:00100000:1.0:1769074838.975746:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66102:x1855007617008384:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:1.0:1769074838.975863:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.975879:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.975890:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.975902:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.975909:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.975915:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.975987:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074838.975998:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074838.976006:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:0.0:1769074838.976131:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00000004:00200000:0.0:1769074838.976136:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074838.976150:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976165:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976172:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976177:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976184:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976190:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.976194:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976213:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976225:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b681697300) 00010000:00010000:0.0:1769074838.976227:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.976232:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976238:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29200/0x7fe2323adfd5526b lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976259:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976265:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976271:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976275:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.976277:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976283:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976290:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b681697100) 00010000:00010000:0.0:1769074838.976292:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.976294:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976299:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55272 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976310:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976317:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976322:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976326:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074838.976328:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976334:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976340:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074838.976343:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074838.976345:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074838.976356:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28000/0x7fe2323adfd55279 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:0.0:1769074838.976455:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66102:x1855007617008384:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 709us (925us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074838.976464:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13515 00000100:00100000:0.0:1769074838.999344:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074838.999422:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617008512 00000100:00100000:2.0:1769074838.999432:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617008512 00000100:00100000:2.0:1769074838.999511:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13516 00000100:00100000:2.0:1769074838.999521:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617008512:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074838.999539:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074838.999561:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55280 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6db expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074838.999623:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.999636:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999648:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999670:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.999677:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999683:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999748:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074838.999756:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999766:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:2.0:1769074838.999843:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00000004:00200000:2.0:1769074838.999848:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074838.999868:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999880:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999889:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999895:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999901:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999909:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999914:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.999918:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999925:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999937:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a000) 00010000:00010000:2.0:1769074838.999940:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074838.999944:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999953:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89000/0x7fe2323adfd55287 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999966:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999972:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999977:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999982:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074838.999985:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999990:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074838.999996:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436ac00) 00010000:00010000:2.0:1769074838.999999:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.000001:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.000006:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5528e lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.000033:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55280 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6db expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.000049:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6db expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.000061:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074839.000063:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074839.000066:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:3.0:1769074839.000272:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66106:x1855007617008512:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 750us (936us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074839.000282:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13516 00000100:00100000:2.0:1769074839.005326:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.005856:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617008640 00000100:00100000:0.0:1769074839.005866:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617008640 00000100:00100000:0.0:1769074839.005923:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14473 00000100:00100000:0.0:1769074839.005933:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617008640:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074839.005951:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074839.005971:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6e2 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074839.005994:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x35:0x0] initial LVB size: 2097152, mtime: 0x6971f096, atime: 0x0, ctime: 0x6971f096, blocks: 0x1000 00010000:00010000:0.0:1769074839.006012:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.006014:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.006024:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074839.006040:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074839.006052:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135a00, rc 0) 00000100:00100000:0.0:1769074839.006105:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617008640:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 173us (789us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.006112:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14473 00000100:00100000:2.0:1769074839.010960:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.011045:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617008896 00000100:00100000:3.0:1769074839.011056:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617008896 00000100:00100000:3.0:1769074839.011117:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13517 00000100:00100000:3.0:1769074839.011128:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66106:x1855007617008896:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074839.011147:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074839.011167:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb000/0x7fe2323adfd552a3 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6e9 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.011294:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd552aa lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074839.011309:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd552aa lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074839.011321:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd552aa lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:3.0:1769074839.011334:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00010000:00010000:3.0:1769074839.011443:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb000/0x7fe2323adfd552a3 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e9 expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.011460:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd552aa lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e9 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.011469:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:3.0:1769074839.011472:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074839.011476:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb200, rc 0) 00000100:00100000:3.0:1769074839.011546:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66106:x1855007617008896:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 414us (590us total) trans 4294968358 rc 0/0 00000100:00100000:3.0:1769074839.011554:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13517 00000100:00100000:2.0:1769074839.018570:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.018731:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617009152 00000100:00100000:3.0:1769074839.018742:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617009152 00000100:00100000:3.0:1769074839.018846:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13518 00000100:00100000:3.0:1769074839.018857:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66106:x1855007617009152:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074839.018883:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074839.018905:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb000/0x7fe2323adfd552b8 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6f0 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.019022:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd552bf lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074839.019038:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd552bf lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074839.019048:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd552bf lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:3.0:1769074839.019060:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00010000:00010000:3.0:1769074839.019164:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb000/0x7fe2323adfd552b8 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f0 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.019182:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd552bf lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f0 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.019191:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:3.0:1769074839.019194:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074839.019208:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbe00, rc 0) 00000100:00100000:3.0:1769074839.019272:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:66106:x1855007617009152:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 417us (713us total) trans 4294968359 rc 0/0 00000100:00100000:3.0:1769074839.019279:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13518 00000100:00100000:2.0:1769074839.023248:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.023351:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617009280 00000100:00100000:1.0:1769074839.023361:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617009280 00000100:00100000:1.0:1769074839.023424:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1044 00000100:00100000:1.0:1769074839.023434:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2409:x1855007617009280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074839.023452:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074839.023476:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd552aa lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e9 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.023504:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.023510:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074839.023574:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617009280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 141us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074839.023581:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1044 00000100:00100000:2.0:1769074839.028336:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.028703:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617009408 00000100:00100000:3.0:1769074839.028715:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617009408 00000100:00100000:3.0:1769074839.028772:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14474 00000100:00100000:3.0:1769074839.028782:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617009408:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074839.028859:0:28519:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.028877:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074839.028884:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074839.028886:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074839.028890:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 4/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.028965:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074839.028980:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 9 pid: 36516 timeout: 1926 lvb_type: 1 00000100:00100000:3.0:1769074839.028994:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625226880:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074839.029322:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074839.034927:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.035043:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625226880:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074839.035070:0:28519:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074839.035149:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617009536 00000100:00100000:0.0:1769074839.035158:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617009536 00010000:00010000:0.0:1769074839.035193:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 10 pid: 36516 timeout: 1926 lvb_type: 1 00000100:00100000:0.0:1769074839.035245:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7207 00000100:00100000:0.0:1769074839.035254:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617009536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074839.035270:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074839.035290:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 10 pid: 36516 timeout: 1926 lvb_type: 1 00010000:00010000:0.0:1769074839.035306:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 10 pid: 36516 timeout: 1926 lvb_type: 1 00010000:00010000:0.0:1769074839.035329:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd5529c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6e2 expref: 10 pid: 36516 timeout: 1926 lvb_type: 1 00010000:00010000:0.0:1769074839.035343:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.035345:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.035349:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.035358:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.035388:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074839.035451:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617009536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 198us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.035459:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7207 00010000:00010000:3.0:1769074839.035784:0:28519:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035798:0:28519:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035854:0:28519:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035863:0:28519:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035871:0:28519:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035878:0:28519:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074839.035883:0:28519:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035891:0:28519:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035903:0:28519:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 2/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.035910:0:28519:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb000/0x7fe2323adfd552cd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074839.035958:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617009408:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 7177us (7631us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074839.035967:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14474 00000100:00100000:2.0:1769074839.039658:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.039861:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617009792 00000100:00100000:1.0:1769074839.039872:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617009792 00000100:00100000:1.0:1769074839.039930:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13519 00000100:00100000:1.0:1769074839.039939:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:66106:x1855007617009792:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:1.0:1769074839.039998:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1769074839.040040:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88200/0x7fe2323adfd55295 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6db expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.040066:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd551c3 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6aa expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.040077:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.040082:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074839.040111:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074839.040124:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.040133:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000040:00080000:1.0:1769074839.040356:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].64, 88 off13296 00000004:00080000:1.0:1769074839.040366:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1826882558820ns, next at 1384406513989ns, rc = 0 00000040:00080000:1.0:1769074839.040398:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].65, 88 off13312 00000004:00080000:1.0:1769074839.040404:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1826882597211ns, next at 1394702964651ns, rc = 0 00000040:00080000:1.0:1769074839.040430:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].66, 88 off13400 00000004:00080000:1.0:1769074839.040435:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1826882628167ns, next at 1394702964651ns, rc = 0 00000001:00080000:3.0:1769074839.043414:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968360 is committed 00000004:00080000:3.0:1769074839.043427:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 64 committed 00000004:00080000:3.0:1769074839.043447:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 63 committed 00000004:00080000:3.0:1769074839.043451:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 62 committed 00000100:00100000:0.0:1769074839.043627:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625227008:0@lo:2:osp-syn-0-1.0 00000100:00100000:0.0:1769074839.043693:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074839.043747:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625227008 00000100:00100000:0.0:1769074839.043753:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625227008 00000100:00100000:0.0:1769074839.043803:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14475 00000100:00100000:0.0:1769074839.043833:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625227008:12345-0@lo:2:osp-syn-0-1.0 00000100:00100000:0.0:1769074839.044005:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+6:3652:x1855007625227008:12345-0@lo:2:osp-syn-0-1.0 Request processed in 172us (310us total) trans 21474841291 rc 0/0 00000100:00100000:0.0:1769074839.044033:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14475 00000100:00100000:0.0:1769074839.044072:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 21474841289 gen 2 00000004:00080000:0.0:1769074839.044077:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2aa00, transno 21474841289 00000100:00100000:0.0:1769074839.044088:0:3652:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b549f29c00 x1855007625227008/t21474841291(21474841291) o2->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074855 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-1.0' uid:0 gid:0 projid:4294967295 00000004:00080000:0.0:1769074839.044098:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f29c00/2, rc 0, transno 21474841291 00000100:00100000:0.0:1769074839.044103:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625227008:0@lo:2:osp-syn-0-1.0 00000100:00100000:2.0:1769074839.044417:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625227136:0@lo:2:osp-syn-1-1.0 00000100:00100000:2.0:1769074839.044459:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074839.044485:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074839.045479:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625227264:0@lo:2:osp-syn-1-1.0 00000100:00100000:2.0:1769074839.045531:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074839.045582:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625227136 00000100:00100000:2.0:1769074839.045588:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625227136 00000100:00100000:2.0:1769074839.045627:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625227264 00000100:00100000:2.0:1769074839.045631:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625227264 00000100:00100000:2.0:1769074839.045651:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14476 00000100:00100000:2.0:1769074839.045658:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+8:3649:x1855007625227136:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:2.0:1769074839.045824:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625227136:12345-0@lo:2:osp-syn-1-1.0 Request processed in 147us (1345us total) trans 17179874615 rc 0/0 00000100:00100000:2.0:1769074839.045830:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14476 00000100:00100000:2.0:1769074839.045844:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14477 00000100:00100000:2.0:1769074839.045848:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+8:3649:x1855007625227264:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:2.0:1769074839.045897:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625227264:12345-0@lo:2:osp-syn-1-1.0 Request processed in 49us (366us total) trans 17179874616 rc 0/0 00000100:00100000:2.0:1769074839.045901:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14477 00000004:00080000:2.0:1769074839.045931:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a271c00/2, rc 0, transno 17179874616 00000100:00100000:2.0:1769074839.045934:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625227264:0@lo:2:osp-syn-1-1.0 00000004:00080000:2.0:1769074839.045940:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a270700/2, rc 0, transno 17179874615 00000100:00100000:2.0:1769074839.045941:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625227136:0@lo:2:osp-syn-1-1.0 00000004:00010000:1.0:1769074839.067047:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000003945a13b reply state 000000000d7e9644 transno 4294968360 ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.067084:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.067092:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.067095:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:1.0:1769074839.067099:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00080000:1.0:1769074839.067239:0:10283:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:1.0:1769074839.067266:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617009792:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 27324us (27614us total) trans 4294968360 rc 0/0 00000100:00100000:1.0:1769074839.067276:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13519 00010000:00010000:3.0:1769074839.067351:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074839.067361:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074839.067368:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074839.067374:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074839.067379:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074839.067387:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074839.067399:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:3.0:1769074839.067401:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074839.067405:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074839.067413:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28a00/0x7fe2323adfd552d4 lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074839.072076:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.072603:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617009920 00000100:00100000:1.0:1769074839.072614:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617009920 00000100:00100000:1.0:1769074839.072681:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13520 00000100:00100000:1.0:1769074839.072692:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66106:x1855007617009920:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074839.072716:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074839.072737:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6f7 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.072796:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.072844:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.072865:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.072868:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074839.072871:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28c00, rc 0) 00000100:00100000:1.0:1769074839.072930:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617009920:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 240us (866us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074839.072938:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13520 00000100:00100000:2.0:1769074839.089619:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.089740:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010048 00000100:00100000:1.0:1769074839.089750:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010048 00000100:00100000:1.0:1769074839.089886:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13521 00000100:00100000:1.0:1769074839.089897:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617010048:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074839.089919:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074839.089941:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28400/0x7fe2323adfd552e2 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c6fe expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.090055:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd552e9 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074839.090070:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd552e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.090082:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd552e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:1.0:1769074839.090109:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097153 00000004:00200000:1.0:1769074839.090114:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074839.090129:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd552e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.090154:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28400/0x7fe2323adfd552e2 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6fe expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.090172:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd552e9 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6fe expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.090181:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.090184:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074839.090187:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29a00, rc 0) 00000100:00100000:1.0:1769074839.090259:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66106:x1855007617010048:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 363us (648us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074839.090266:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13521 00000100:00100000:2.0:1769074839.099209:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.099338:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010176 00000100:00100000:0.0:1769074839.099347:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010176 00000100:00100000:0.0:1769074839.099432:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14478 00000100:00100000:0.0:1769074839.099443:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617010176:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074839.099462:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074839.099483:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c705 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074839.099520:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x35:0x0] initial LVB size: 2097152, mtime: 0x6971f096, atime: 0x0, ctime: 0x6971f096, blocks: 0x1000 00010000:00010000:0.0:1769074839.099540:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.099553:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.099563:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074839.099580:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 9 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074839.099593:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135a00, rc 0) 00000100:00100000:0.0:1769074839.099659:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617010176:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 218us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.099668:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14478 00000100:00100000:2.0:1769074839.186940:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.187103:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010304 00000100:00100000:0.0:1769074839.187113:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010304 00000100:00100000:0.0:1769074839.187177:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14479 00000100:00100000:0.0:1769074839.187187:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617010304:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074839.187381:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617010304:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 194us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.187400:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14479 00000100:00100000:1.0:1769074839.203323:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.203432:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010432 00000100:00100000:2.0:1769074839.203443:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010432 00000100:00100000:2.0:1769074839.203505:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14480 00000100:00100000:2.0:1769074839.203515:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617010432:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:2.0:1769074839.203622:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617010432:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 109us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074839.203630:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14480 00000100:00100000:1.0:1769074839.240565:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.240678:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010560 00000100:00100000:2.0:1769074839.240690:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010560 00000100:00100000:2.0:1769074839.240766:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8745 00000100:00100000:2.0:1769074839.240776:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617010560:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:2.0:1769074839.240847:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.240866:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074839.240869:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074839.240873:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.240887:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290314:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290337:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290346:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290353:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074839.290357:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290366:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290401:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.290408:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd552f7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074839.290499:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617010560:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 49718us (49938us total) trans 0 rc 2097152/2097152 00000100:00100000:2.0:1769074839.290510:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8745 00000100:00100000:1.0:1769074839.311937:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.312081:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010688 00000100:00100000:2.0:1769074839.312092:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010688 00000100:00100000:2.0:1769074839.312249:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8746 00000100:00100000:2.0:1769074839.312261:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617010688:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:2.0:1769074839.312351:0:28434:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x35:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1769074839.312361:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.312376:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074839.312379:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074839.312384:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.312400:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00080000:0.0:1769074839.312550:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:3.0:1769074839.365269:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874617 is committed 00000001:00080000:3.0:1769074839.365289:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874616 is committed 00002000:00010000:0.0:1769074839.365442:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1769074839.365447:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb mtime from disk: 0 -> 1769074838 00002000:00010000:0.0:1769074839.365449:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb ctime from disk: 0 -> 1769074838 00002000:00010000:0.0:1769074839.365451:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:0.0:1769074839.365465:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.365477:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.365486:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.365493:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074839.365498:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.365507:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.365529:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.365550:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd552fe lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074839.365660:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617010688:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 53396us (53727us total) trans 17179874617 rc 0/0 00000100:00100000:0.0:1769074839.365672:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8746 00000100:00100000:3.0:1769074839.374207:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.374380:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617010816 00000100:00100000:0.0:1769074839.374392:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617010816 00000100:00100000:0.0:1769074839.374472:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8747 00000100:00100000:0.0:1769074839.374483:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007617010816:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:0.0:1769074839.374576:0:28434:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x36:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1769074839.374585:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.374599:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.374601:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.374607:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.374621:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000001:00080000:3.0:1769074839.406451:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874618 is committed 00002000:00010000:1.0:1769074839.406631:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:1.0:1769074839.406636:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb mtime from disk: 0 -> 1769074839 00002000:00010000:1.0:1769074839.406639:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb ctime from disk: 0 -> 1769074839 00002000:00010000:1.0:1769074839.406642:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:1.0:1769074839.406655:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.406668:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.406678:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.406686:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074839.406690:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.406699:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.406719:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.406726:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549134800/0x7fe2323adfd55305 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074839.407139:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007617010816:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 32652us (32947us total) trans 17179874618 rc 0/0 00000100:00100000:1.0:1769074839.407151:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8747 00000100:00100000:2.0:1769074839.477615:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.477865:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011072 00000100:00100000:1.0:1769074839.477875:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011072 00000100:00100000:1.0:1769074839.477947:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13522 00000100:00100000:1.0:1769074839.477957:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66106:x1855007617011072:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:1.0:1769074839.478019:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074839.478050:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd552e9 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6fe expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.478069:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.478073:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074839.478268:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074839.478284:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.478294:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000001:00080000:1.0:1769074839.478335:0:10283:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 4294968361 00000004:00010000:1.0:1769074839.478356:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000007c86dcab reply state 00000000d5489a62 transno 4294968362 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.478368:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074839.478374:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.478377:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1769074839.478380:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074839.480465:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66106:x1855007617011072:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 2496us (2847us total) trans 4294968362 rc 0/0 00000100:00100000:1.0:1769074839.480477:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13522 00000020:00080000:1.0:1769074839.480485:0:10283:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1769074839.489611:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.489718:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011200 00000100:00100000:3.0:1769074839.489728:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011200 00000100:00100000:3.0:1769074839.489789:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14481 00000100:00100000:3.0:1769074839.489799:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66106:x1855007617011200:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074839.489848:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074839.489870:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c70c expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074839.489893:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x35:0x0] initial LVB size: 1048576, mtime: 0x6971f096, atime: 0x0, ctime: 0x6971f096, blocks: 0x800 00010000:00010000:3.0:1769074839.489913:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074839.489915:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074839.489924:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.489939:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.489950:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb000, rc 0) 00000100:00100000:3.0:1769074839.490003:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617011200:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 205us (403us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074839.490009:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14481 00000100:00100000:2.0:1769074839.492031:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.492076:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.492157:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011456 00000100:00100000:1.0:1769074839.492167:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011456 00000100:00100000:1.0:1769074839.492237:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2430 00000100:00100000:1.0:1769074839.492245:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007617011456:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074839.492322:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2406:x1855007617011456:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 78us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074839.492329:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2430 00000100:00100000:3.0:1769074839.492582:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011328 00000100:00100000:3.0:1769074839.492587:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011328 00000100:00100000:3.0:1769074839.492626:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 369 00000100:00100000:3.0:1769074839.492633:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649911050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617011328:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074839.492664:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649911050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617011328:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 31us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074839.492671:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 369 00000100:00100000:1.0:1769074839.494370:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.494469:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011584 00000100:00100000:0.0:1769074839.494479:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011584 00000100:00100000:0.0:1769074839.494530:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14482 00000100:00100000:0.0:1769074839.494537:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617011584:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074839.494555:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074839.494575:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c713 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074839.494597:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x36:0x0] initial LVB size: 2097152, mtime: 0x6971f097, atime: 0x0, ctime: 0x6971f097, blocks: 0x800 00010000:00010000:0.0:1769074839.494617:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.494619:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074839.494627:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 9 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.494642:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 9 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.494654:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134600, rc 0) 00000100:00100000:0.0:1769074839.494707:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66106:x1855007617011584:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 170us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.494713:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14482 00000100:00100000:2.0:1769074839.499882:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.500401:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011712 00000100:00100000:0.0:1769074839.500413:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011712 00002000:00010000:0.0:1769074839.500457:0:28434:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161088/54 (2097152->18446744073709551615) 00002000:00100000:0.0:1769074839.500462:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a270000 with x1855007617011712 ext(2097152->18446744073709551615) 00002000:00010000:0.0:1769074839.500475:0:28434:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a270000. 00000100:00100000:0.0:1769074839.500517:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8748 00000100:00100000:0.0:1769074839.500527:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617011712:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:0.0:1769074839.500691:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074839.500695:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb mtime from disk: 1769074839 -> 1769074839 00002000:00010000:0.0:1769074839.500698:0:28434:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074839.500701:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb ctime from disk: 1769074839 -> 1769074839 00002000:00010000:0.0:1769074839.500704:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074839.500779:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617011712:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 253us (906us total) trans 17179874619 rc 0/0 00000100:00100000:0.0:1769074839.500788:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8748 00002000:00010000:0.0:1769074839.500793:0:28434:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161088/54 (2097152->18446744073709551615) 00002000:00100000:0.0:1769074839.500795:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b67a270000 with x1855007617011712 ext(2097152->18446744073709551615) 00002000:00010000:0.0:1769074839.500800:0:28434:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b67a270000. 00000100:00100000:2.0:1769074839.506247:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.506351:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617011968 00000100:00100000:0.0:1769074839.506362:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617011968 00000100:00100000:0.0:1769074839.506427:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13523 00000100:00100000:0.0:1769074839.506438:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66106:x1855007617011968:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:0.0:1769074839.506557:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.506575:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.506586:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00010000:0.0:1769074839.506692:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000de2ce9f6 reply state 000000004b4fd845 transno 4294968363 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.506705:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.506712:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.506715:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1769074839.506719:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1769074839.506841:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:66106:x1855007617011968:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 369us (568us total) trans 4294968363 rc 0/0 00000100:00100000:0.0:1769074839.506849:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13523 00000100:00100000:1.0:1769074839.510579:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.510669:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617012096 00000100:00100000:3.0:1769074839.510680:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617012096 00000100:00100000:3.0:1769074839.510738:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14483 00000100:00100000:3.0:1769074839.510748:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66106:x1855007617012096:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074839.510766:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074839.510784:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c71a expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.510869:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074839.510877:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074839.510879:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074839.510883:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->10485759] (req 0->10485759) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.510916:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074839.510929:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 1927 lvb_type: 1 00000100:00100000:3.0:1769074839.510942:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54538bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625227392:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074839.511036:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266a80 going to sleep for 11 seconds 00000100:00100000:2.0:1769074839.519258:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074839.519303:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54538bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625227392:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074839.519349:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->10485759] (req 0->10485759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074839.519370:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febac00, rc 0) 00000100:00100000:3.0:1769074839.519451:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66106:x1855007617012096:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 8704us (8882us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074839.519461:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14483 00000100:00100000:1.0:1769074839.519521:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617012224 00000100:00100000:1.0:1769074839.519528:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617012224 00010000:00010000:1.0:1769074839.519562:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 1927 lvb_type: 1 00000100:00100000:1.0:1769074839.519595:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7208 00000100:00100000:1.0:1769074839.519604:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617012224:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074839.519620:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074839.519642:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 1927 lvb_type: 1 00010000:00010000:1.0:1769074839.519654:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 1927 lvb_type: 1 00010000:00010000:1.0:1769074839.519677:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549135a00/0x7fe2323adfd552f0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c705 expref: 10 pid: 36516 timeout: 1927 lvb_type: 1 00010000:00010000:1.0:1769074839.519696:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074839.519698:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074839.519706:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.519716:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074839.519751:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074839.519765:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625227520:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074839.519872:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b665604c00 going to sleep for 11 seconds 00000100:00100000:1.0:1769074839.523404:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625227520:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074839.523433:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074839.523501:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617012224:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 3897us (4254us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074839.523513:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7208 00000100:00100000:3.0:1769074839.523662:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.524487:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617012352 00000100:00100000:0.0:1769074839.524497:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617012352 00000100:00100000:0.0:1769074839.524558:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14484 00000100:00100000:0.0:1769074839.524567:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617012352:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074839.524734:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074839.524738:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb mtime from disk: 1769074838 -> 1769074838 00002000:00010000:0.0:1769074839.524740:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb atime from disk: 0 -> 1769074838 00002000:00010000:0.0:1769074839.524743:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb ctime from disk: 1769074838 -> 1769074839 00002000:00010000:0.0:1769074839.524745:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074839.524800:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617012352:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 233us (1138us total) trans 21474841292 rc 0/0 00000100:00100000:0.0:1769074839.525235:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14484 00000100:00100000:3.0:1769074839.531375:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.531684:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617012608 00000100:00100000:2.0:1769074839.532144:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617012608 00000100:00100000:2.0:1769074839.532226:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4734 00000100:00100000:2.0:1769074839.532235:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:66106:x1855007617012608:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:2.0:1769074839.532298:0:15947:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x240000404:0x4b:0x0] lease broken? 0 ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd552bf lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f0 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.532346:0:15947:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074839.532358:0:15947:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 2/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.532368:0:15947:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 4/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074839.532399:0:15947:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.532412:0:15947:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 4/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 13 pid: 10283 timeout: 1927 lvb_type: 0 00000100:00100000:2.0:1769074839.532425:0:15947:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0001_UUID:15947:1855007625227648:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074839.532522:0:15947:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b4523c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074839.537145:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074839.537262:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617012736 00000100:00100000:1.0:1769074839.537273:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617012736 00010000:00010000:1.0:1769074839.537315:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 5/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 14 pid: 10283 timeout: 1927 lvb_type: 0 00000100:00100000:1.0:1769074839.537361:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7209 00000100:00100000:1.0:1769074839.537371:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effe050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2408:x1855007617012736:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074839.537389:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074839.537395:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 5/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 14 pid: 10283 timeout: 1927 lvb_type: 0 00010000:00010000:1.0:1769074839.537409:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 4/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 14 pid: 10283 timeout: 1927 lvb_type: 0 00010000:00010000:1.0:1769074839.537432:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074839.537438:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:1.0:1769074839.537449:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 15947 initiator: MDT1 00010000:00010000:1.0:1769074839.537459:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 4/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 15947 initiator: MDT1 00010000:00010000:1.0:1769074839.537480:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074839.537549:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effe050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2408:x1855007617012736:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 179us (414us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074839.537556:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7209 00010000:00010000:0.0:1769074839.537855:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a28c00/0x7fe2323adfd552db lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f7 expref: 13 pid: 10283 timeout: 1927 lvb_type: 0 00000100:00100000:0.0:1769074839.537875:0:15947:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0001_UUID:15947:1855007625227648:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074839.537895:0:15947:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.537904:0:15947:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.537910:0:15947:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00000040:00080000:0.0:1769074839.538140:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].65, 88 off13384 00000004:00080000:0.0:1769074839.538148:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1827380340427ns, next at 1384406513989ns, rc = 0 00000040:00080000:0.0:1769074839.538177:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].67, 88 off13488 00000004:00080000:0.0:1769074839.538182:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1827380375581ns, next at 1394702964651ns, rc = 0 00000040:00080000:0.0:1769074839.538341:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].68, 88 off13576 00000004:00080000:0.0:1769074839.538346:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1827380539289ns, next at 1394702964651ns, rc = 0 00000001:00080000:3.0:1769074839.545249:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968364 is committed 00000004:00080000:3.0:1769074839.545263:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 66 committed 00000004:00080000:3.0:1769074839.545277:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 65 committed 00000004:00080000:3.0:1769074839.545280:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 63 committed 00010000:00010000:2.0:1769074839.545369:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545382:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545390:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545398:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.545403:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545412:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545429:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074839.545432:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.545437:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545443:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd5530c lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.545463:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.545470:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.545475:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.545480:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.545482:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.545488:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.545495:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074839.545497:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.545501:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074839.545506:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55321 lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:2.0:1769074839.545715:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625227776:0@lo:2:osp-syn-0-1.0 00000100:00100000:2.0:1769074839.545778:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00010000:0.0:1769074839.545991:0:15947:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000b12298af reply state 00000000698a10a6 transno 4294968364 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00000100:00100000:2.0:1769074839.546208:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074839.546218:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625228032:0@lo:2:osp-syn-1-1.0 00000100:00100000:2.0:1769074839.546249:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074839.546279:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074839.546286:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625227904:0@lo:2:osp-syn-1-1.0 00000100:00100000:2.0:1769074839.546305:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074839.546331:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625227776 00000100:00100000:2.0:1769074839.546338:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625227776 00000100:00100000:2.0:1769074839.546377:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228032 00000100:00100000:2.0:1769074839.546379:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228032 00000100:00100000:2.0:1769074839.546395:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625227904 00000100:00100000:2.0:1769074839.546397:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625227904 00000100:00100000:2.0:1769074839.546417:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14485 00000100:00100000:2.0:1769074839.546425:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625227776:12345-0@lo:2:osp-syn-0-1.0 00002000:00010000:2.0:1769074839.546572:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074839.546576:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb mtime from disk: 1769074838 -> 1769074838 00002000:00010000:2.0:1769074839.546578:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb atime from disk: 1769074838 -> 1769074838 00002000:00010000:2.0:1769074839.546581:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb ctime from disk: 1769074839 -> 1769074839 00002000:00010000:2.0:1769074839.546584:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x35:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:2.0:1769074839.546618:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625227776:12345-0@lo:2:osp-syn-0-1.0 Request processed in 193us (839us total) trans 21474841293 rc 0/0 00000100:00100000:2.0:1769074839.546630:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14485 00000100:00100000:2.0:1769074839.546646:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14486 00000100:00100000:2.0:1769074839.546651:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3650:x1855007625228032:12345-0@lo:2:osp-syn-1-1.0 00002000:00010000:2.0:1769074839.547177:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074839.547182:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb mtime from disk: 1769074839 -> 1769074839 00002000:00010000:2.0:1769074839.547184:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074839.547196:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb ctime from disk: 1769074839 -> 1769074839 00002000:00010000:2.0:1769074839.547199:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x36:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074839.547239:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625228032:12345-0@lo:2:osp-syn-1-1.0 Request processed in 586us (988us total) trans 17179874620 rc 0/0 00000100:00100000:2.0:1769074839.547246:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14486 00000100:00100000:2.0:1769074839.547259:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14487 00000100:00100000:2.0:1769074839.547266:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625227904:12345-0@lo:2:osp-syn-1-1.0 00002000:00010000:2.0:1769074839.547317:0:8398:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:2.0:1769074839.547320:0:8398:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb mtime from disk: 1769074838 -> 1769074838 00002000:00010000:2.0:1769074839.547323:0:8398:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074839.547325:0:8398:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb ctime from disk: 1769074838 -> 1769074838 00002000:00010000:2.0:1769074839.547327:0:8398:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x35:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074839.547352:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625227904:12345-0@lo:2:osp-syn-1-1.0 Request processed in 87us (1046us total) trans 17179874621 rc 0/0 00000100:00100000:2.0:1769074839.547358:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14487 00000100:00100000:2.0:1769074839.547403:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 17179874616 gen 2 00000004:00080000:2.0:1769074839.547408:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6656d9c00, transno 17179874613 00000004:00080000:2.0:1769074839.547418:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6656daa00, transno 17179874614 00000004:00080000:2.0:1769074839.547421:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a270700, transno 17179874615 00000004:00080000:2.0:1769074839.547424:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a271c00, transno 17179874616 00000100:00100000:2.0:1769074839.547431:0:3650:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b549f28a80 x1855007625228032/t17179874620(17179874620) o2->lustre-OST0001-osc-MDT0001@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074855 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-1.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1769074839.547726:0:15947:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.547738:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.547742:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:0.0:1769074839.547746:0:15947:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1769074839.547753:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.547756:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1769074839.547763:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd552bf lrc: 0/0,0 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6f0 expref: 7 pid: 6512 timeout: 0 lvb_type: 0 00010000:00080000:0.0:1769074839.547865:0:15947:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:0.0:1769074839.547878:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66106:x1855007617012608:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 15643us (16514us total) trans 4294968364 rc 0/0 00000100:00100000:0.0:1769074839.547887:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4734 00010000:00010000:0.0:1769074839.548172:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.548181:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.548200:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.548205:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.548209:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.548216:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.548226:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.548228:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074839.548231:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074839.548236:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd5532f lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT1 00000004:00080000:2.0:1769074839.549201:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f28a80/2, rc 0, transno 17179874620 00000100:00100000:2.0:1769074839.549210:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625228032:0@lo:2:osp-syn-1-1.0 00000004:00080000:0.0:1769074839.549434:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2aa00/2, rc 0, transno 17179874621 00000100:00100000:0.0:1769074839.549441:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625227904:0@lo:2:osp-syn-1-1.0 00000004:00080000:0.0:1769074839.549467:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5444a6a00/2, rc 0, transno 21474841293 00000100:00100000:0.0:1769074839.549471:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625227776:0@lo:2:osp-syn-0-1.0 00000004:00080000:0.0:1769074839.549638:0:9490:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0001: 0 changes, 2 in progress, 0 in flight, 4 done 00000100:00100000:1.0:1769074839.552441:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.552520:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617012864 00000100:00100000:2.0:1769074839.552530:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617012864 00000100:00100000:2.0:1769074839.552596:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13524 00000100:00100000:2.0:1769074839.552606:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66106:x1855007617012864:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074839.552628:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074839.552647:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55336 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c721 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.552703:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55336 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c721 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.552718:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55336 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c721 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.552737:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074839.552740:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074839.552744:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88600, rc 0) 00000100:00100000:2.0:1769074839.552802:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66106:x1855007617012864:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 198us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074839.552831:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13524 00000100:00100000:0.0:1769074839.561423:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.566302:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617013120 00000100:00100000:2.0:1769074839.566323:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617013120 00000100:00100000:2.0:1769074839.566403:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4735 00000100:00100000:2.0:1769074839.566415:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66106:x1855007617013120:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074839.566701:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66106:x1855007617013120:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 287us (5289us total) trans 4294968365 rc 0/0 00000100:00100000:2.0:1769074839.566709:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4735 00000100:00100000:3.0:1769074839.630329:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.630568:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617013376 00000100:00100000:0.0:1769074839.630578:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617013376 00000100:00100000:0.0:1769074839.630647:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13525 00000100:00100000:0.0:1769074839.630658:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66112:x1855007617013376:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074839.630685:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074839.630708:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5533d lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c728 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.630785:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.630799:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.630857:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.630872:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.630879:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.630884:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.630964:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.630971:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631107:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631113:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631118:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.631122:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631129:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631141:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317700) 00010000:00010000:0.0:1769074839.631144:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.631149:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631155:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135400/0x7fe2323adfd55344 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631168:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631174:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631180:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631196:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.631199:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631205:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631212:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317000) 00010000:00010000:0.0:1769074839.631215:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.631218:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631224:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd5534b lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.631259:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5533d lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c728 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.631269:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317700) 00010000:00010000:0.0:1769074839.631271:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074839.631275:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5533d lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c728 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.631286:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135600, rc 0) 00000100:00100000:0.0:1769074839.631350:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66112:x1855007617013376:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 694us (1032us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074839.631358:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13525 00000100:00100000:3.0:1769074839.636355:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.638743:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617013504 00000100:00100000:0.0:1769074839.638758:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617013504 00000100:00100000:0.0:1769074839.638852:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13526 00000100:00100000:0.0:1769074839.638862:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66112:x1855007617013504:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074839.638935:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.638951:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.638964:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.638976:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.638984:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.638991:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639062:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.639071:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639078:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:0.0:1769074839.639119:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097152 00000004:00200000:0.0:1769074839.639123:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074839.639135:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639149:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639155:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639160:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639166:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639172:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.639176:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639183:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639205:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317300) 00010000:00010000:0.0:1769074839.639208:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.639213:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639219:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd55352 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639238:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639244:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639249:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639254:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.639257:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639262:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639269:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317b00) 00010000:00010000:0.0:1769074839.639271:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.639273:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639277:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55359 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639287:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639293:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639299:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639304:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.639306:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639312:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639319:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.639321:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.639324:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.639329:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55360 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:0.0:1769074839.639418:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66112:x1855007617013504:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 556us (3073us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.639426:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13526 00000100:00100000:3.0:1769074839.689219:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.690252:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617013760 00000100:00100000:2.0:1769074839.690264:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617013760 00000100:00100000:2.0:1769074839.690336:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13527 00000100:00100000:2.0:1769074839.690346:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66117:x1855007617013760:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074839.690366:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074839.690391:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55367 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c72f expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.690469:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.690484:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690495:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690507:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.690514:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690521:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690593:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690602:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690608:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690615:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690621:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.690624:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690632:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690644:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.690647:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.690651:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690657:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5536e lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690670:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690675:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690680:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690685:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.690688:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690694:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690700:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436a500) 00010000:00010000:2.0:1769074839.690703:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.690705:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690711:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55375 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.690742:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55367 lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c72f expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.690752:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.690754:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074839.690758:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55367 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c72f expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.690768:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074839.690883:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66117:x1855007617013760:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 538us (1675us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074839.690897:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13527 00000100:00100000:3.0:1769074839.700266:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.700386:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617013888 00000100:00100000:0.0:1769074839.700397:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617013888 00000100:00100000:0.0:1769074839.700461:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13528 00000100:00100000:0.0:1769074839.700472:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66117:x1855007617013888:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074839.700558:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.700579:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700591:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700603:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.700609:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700616:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700682:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.700692:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700699:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:0.0:1769074839.700736:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097152 00000004:00200000:0.0:1769074839.700741:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074839.700752:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700764:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700771:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700782:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700789:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700795:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.700799:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700852:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700866:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317b00) 00010000:00010000:0.0:1769074839.700869:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.700874:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700880:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5537c lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700899:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700905:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700911:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700917:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.700919:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700929:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700936:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317300) 00010000:00010000:0.0:1769074839.700939:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.700941:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700947:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd55383 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700958:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700964:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700970:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700976:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.700978:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700984:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.700990:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.700992:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.700995:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.701018:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd5538a lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:0.0:1769074839.701091:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66117:x1855007617013888:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 620us (833us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.701100:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13528 00000100:00100000:3.0:1769074839.743491:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.743600:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617014144 00000100:00100000:2.0:1769074839.743611:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617014144 00000100:00100000:2.0:1769074839.743674:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13529 00000100:00100000:2.0:1769074839.743684:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66124:x1855007617014144:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074839.743711:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074839.743740:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55391 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c736 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.743866:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.743882:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.743893:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.743907:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.743914:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.743920:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.743990:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.743997:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744002:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744008:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744026:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.744030:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744037:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744048:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.744051:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.744055:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744060:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55398 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744073:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744078:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744083:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744088:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.744090:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744095:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744101:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436a500) 00010000:00010000:2.0:1769074839.744103:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.744105:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744114:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5539f lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.744144:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55391 lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c736 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.744154:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.744156:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074839.744159:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55391 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c736 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.744169:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074839.744245:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66124:x1855007617014144:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 562us (763us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074839.744252:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13529 00000100:00100000:3.0:1769074839.753851:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.757521:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617014272 00000100:00100000:0.0:1769074839.757537:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617014272 00000100:00100000:0.0:1769074839.757615:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13530 00000100:00100000:0.0:1769074839.757625:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66124:x1855007617014272:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074839.757703:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.757719:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757730:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757740:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.757746:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757752:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757839:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.757848:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757855:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:0.0:1769074839.757896:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097152 00000004:00200000:0.0:1769074839.757900:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074839.757913:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757926:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757932:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757937:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757943:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757948:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.757951:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757958:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757969:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317300) 00010000:00010000:0.0:1769074839.757971:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.757976:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757981:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553a6 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.757998:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758003:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758022:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758027:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.758030:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758034:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758040:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317b00) 00010000:00010000:0.0:1769074839.758042:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.758045:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758049:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ad lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758059:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758064:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758068:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758072:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.758074:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758083:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758089:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:0.0:1769074839.758090:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.758093:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.758097:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553b4 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:0.0:1769074839.758193:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66124:x1855007617014272:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 559us (4342us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074839.758201:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13530 00000100:00100000:3.0:1769074839.835631:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.835863:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617014528 00000100:00100000:2.0:1769074839.835874:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617014528 00000100:00100000:2.0:1769074839.835937:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13531 00000100:00100000:2.0:1769074839.835947:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66132:x1855007617014528:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074839.835965:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074839.835988:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553bb lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c73d expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.836061:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.836075:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836090:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836103:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.836110:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836115:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836195:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836202:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836207:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836213:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836218:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.836222:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836228:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836240:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.836243:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.836248:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836253:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553c2 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836268:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836273:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836279:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836283:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.836286:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836292:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836299:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436a500) 00010000:00010000:2.0:1769074839.836302:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.836304:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836310:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553c9 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.836344:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553bb lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c73d expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.836354:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.836357:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074839.836360:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553bb lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c73d expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074839.836371:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074839.836428:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66132:x1855007617014528:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 482us (806us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074839.836436:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13531 00000100:00100000:3.0:1769074839.849224:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.849451:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617014656 00000100:00100000:2.0:1769074839.849462:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617014656 00000100:00100000:2.0:1769074839.849528:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13532 00000100:00100000:2.0:1769074839.849547:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66132:x1855007617014656:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074839.849624:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.849643:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849654:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849665:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.849671:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849677:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849748:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.849761:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849768:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:2.0:1769074839.849839:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097152 00000004:00200000:2.0:1769074839.849844:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074839.849856:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849869:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849875:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849881:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849888:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849894:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.849898:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849905:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849916:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a100) 00010000:00010000:2.0:1769074839.849918:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.849924:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849930:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd553d0 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849947:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849953:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849958:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849963:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.849965:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849971:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849978:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436a500) 00010000:00010000:2.0:1769074839.849980:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.849983:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.849987:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd553d7 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850008:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850014:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850022:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850026:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.850029:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850034:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850041:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074839.850043:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.850045:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.850050:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553de lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074839.850124:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66132:x1855007617014656:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 577us (909us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074839.850132:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13532 00000100:00100000:3.0:1769074839.877062:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074839.877205:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617014912 00000100:00100000:0.0:1769074839.877216:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617014912 00000100:00100000:0.0:1769074839.877280:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13533 00000100:00100000:0.0:1769074839.877290:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66135:x1855007617014912:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074839.877309:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074839.877333:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553e5 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c744 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.877412:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.877427:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877452:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877469:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074839.877476:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877481:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877554:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877562:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877568:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877574:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877579:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.877583:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877589:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877600:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317b00) 00010000:00010000:0.0:1769074839.877603:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.877608:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877613:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134e00/0x7fe2323adfd553ec lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877628:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877634:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877639:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877645:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074839.877648:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877653:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877659:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b67a317300) 00010000:00010000:0.0:1769074839.877662:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074839.877665:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877671:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134c00/0x7fe2323adfd553f3 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074839.877702:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553e5 lrc: 1/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c744 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.877711:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b67a317b00) 00010000:00010000:0.0:1769074839.877713:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074839.877716:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd553e5 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c744 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074839.877732:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135e00, rc 0) 00000100:00100000:0.0:1769074839.877793:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66135:x1855007617014912:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 503us (741us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074839.877800:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13533 00000100:00100000:3.0:1769074839.885592:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074839.885746:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015040 00000100:00100000:2.0:1769074839.885757:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015040 00000100:00100000:2.0:1769074839.885853:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13534 00000100:00100000:2.0:1769074839.885863:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66135:x1855007617015040:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074839.885941:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.885956:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.885970:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.885982:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.885987:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.885992:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886072:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074839.886080:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886086:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:2.0:1769074839.886125:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097152 00000004:00200000:2.0:1769074839.886128:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074839.886139:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886150:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886156:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886161:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886170:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886175:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.886179:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886212:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886224:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b65436a500) 00010000:00010000:2.0:1769074839.886227:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.886231:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886237:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd553fa lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886254:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886259:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886265:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886269:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.886272:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886277:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886288:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b65436a100) 00010000:00010000:2.0:1769074839.886290:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.886293:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886297:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55401 lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886307:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886313:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886317:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886321:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074839.886323:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886329:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886335:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074839.886337:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074839.886340:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074839.886345:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55408 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074839.886428:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66135:x1855007617015040:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 566us (845us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074839.886436:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13534 00010000:00010000:3.0:1769074839.966326:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074840.477098:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074840.477131:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625228160:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074840.477213:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074840.477342:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228160 00000100:00100000:3.0:1769074840.477353:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228160 00000100:00100000:3.0:1769074840.477412:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2709 00000100:00100000:3.0:1769074840.477422:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625228160:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074840.477455:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446146048 avail 1299906560 00000100:00100000:3.0:1769074840.477510:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625228160:12345-0@lo:41:osp-pre-1-0.0 Request processed in 90us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074840.477517:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2709 00000100:00100000:1.0:1769074840.477619:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968350 gen 2 00000004:00080000:1.0:1769074840.477625:0:3649:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968350 committed transno 4294968350 00000100:00100000:1.0:1769074840.477656:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625228160:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074840.478457:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074840.478469:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625228288:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074840.478509:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074840.478582:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228288 00000100:00100000:0.0:1769074840.478588:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228288 00000100:00100000:0.0:1769074840.478632:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2710 00000100:00100000:0.0:1769074840.478640:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625228288:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074840.478663:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445392384 avail 1299152896 00000100:00100000:0.0:1769074840.478717:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625228288:12345-0@lo:41:osp-pre-0-1.0 Request processed in 79us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074840.478724:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2710 00000100:00100000:1.0:1769074840.478856:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625228288:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074840.989260:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625228416:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074840.989329:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074840.989384:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228416 00000100:00100000:1.0:1769074840.989392:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228416 00000100:00100000:1.0:1769074840.989443:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1220 00000100:00100000:1.0:1769074840.989452:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625228416:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074840.989477:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074840.989525:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625228416:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074840.989532:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1220 00000100:00100000:1.0:1769074840.990082:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625228416:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1769074840.990181:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074840.990941:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625228672:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074840.990997:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074840.991068:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074840.991075:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625228544:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074840.991099:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1769074840.991529:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074840.991976:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228672 00000100:00100000:0.0:1769074840.992773:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228672 00000100:00080000:2.0:1769074840.992841:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074840.992865:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228544 00000100:00100000:0.0:1769074840.992868:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228544 00000100:00080000:2.0:1769074840.992895:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074840.992899:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1221 00000100:00100000:0.0:1769074840.992908:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625228672:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074840.992931:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3912896512 avail 3681374208 00000100:00100000:3.0:1769074840.993151:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1222 00000100:00100000:3.0:1769074840.993160:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625228544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769074840.993183:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 1073152 granted 23426368 pending 0 free 3917004800 avail 3683336192 00000100:00100000:0.0:1769074840.993298:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625228672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 392us (2300us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074840.993307:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1221 00000100:00100000:2.0:1769074841.002973:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625228672:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074841.011332:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625228544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 18163us (20221us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074841.011346:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1222 00000100:00100000:2.0:1769074841.011673:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625228544:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074841.501060:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625228800:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074841.501134:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074841.501251:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228800 00000100:00100000:3.0:1769074841.501260:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228800 00000100:00100000:3.0:1769074841.501350:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1223 00000100:00100000:3.0:1769074841.501360:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625228800:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074841.501392:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 1073152 granted 23426368 pending 0 free 3914907648 avail 3683385344 00000100:00080000:1.0:1769074841.501957:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074841.506418:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625228800:12345-0@lo:13:osp-pre-1-1.0 Request processed in 5050us (5274us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074841.506451:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1223 00000100:00100000:2.0:1769074841.510344:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625228800:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074842.016900:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1769074842.026100:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944097 is committed 00000020:00080000:3.0:1769074842.026182:0:32377:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff89b549266b40 refcount 2 committed 0 result 0 batchid 4294969366 00000020:00080000:3.0:1769074842.026189:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67b702c00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b5492660c0 00000020:00080000:3.0:1769074842.026192:0:32377:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000405:0x2:0x0].6 00000020:00080000:3.0:1769074842.026195:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67b702500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff89b549255c00 00000020:00080000:3.0:1769074842.026198:0:32377:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].84 00000020:00080000:3.0:1769074842.026213:0:32377:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 4294969365 batchid updated 00000020:00080000:3.0:1769074842.026234:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294969366 committed batchid 4294969365 00000020:00080000:3.0:1769074842.026362:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294969365 cancel update log [0x200000bd0:0x2:0x0].83: rc = 0 00000020:00080000:3.0:1769074842.026371:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969365 00010000:00010000:2.0:1769074842.026529:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026542:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026551:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026558:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074842.026564:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026571:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026587:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x84e5157e (ffff89b681690f00) 00010000:00010000:2.0:1769074842.026591:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074842.026596:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026602:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd550d5 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x84e5157e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026623:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026630:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026635:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026639:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074842.026642:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026648:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026655:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x205:0x0].0x0 (ffff89b681690e00) 00010000:00010000:2.0:1769074842.026658:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074842.026661:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026666:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd550dc lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x205:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026675:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026680:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026690:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026694:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074842.026697:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026703:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026710:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x7:0x0].0x0 (ffff89b681690600) 00010000:00010000:2.0:1769074842.026712:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074842.026715:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074842.026720:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd550e3 lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074843.043633:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074843.550224:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625228928:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550301:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074843.550385:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625229440:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550414:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074843.550433:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625229952:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550456:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074843.550471:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625230464:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550489:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074843.550520:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1769074843.550526:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625229312:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550548:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074843.550579:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625229824:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550599:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074843.550613:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625230336:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.550646:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.550852:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625229056:0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.550886:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.550903:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625229568:0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.550921:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.550935:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625230080:0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.550953:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.550970:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074843.550977:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625229184:0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.550998:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.551012:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625229696:0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551033:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.551044:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625230208:0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551067:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074843.551100:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229952 00000100:00100000:2.0:1769074843.551108:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229952 00000100:00100000:2.0:1769074843.551165:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625230464 00000100:00100000:2.0:1769074843.551168:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625230464 00000100:00100000:2.0:1769074843.551180:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229824 00000100:00100000:2.0:1769074843.551183:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229824 00000100:00100000:2.0:1769074843.551194:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229568 00000100:00100000:2.0:1769074843.551196:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229568 00000100:00100000:2.0:1769074843.551205:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229184 00000100:00100000:2.0:1769074843.551207:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229184 00000100:00100000:2.0:1769074843.551215:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229696 00000100:00100000:2.0:1769074843.551217:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229696 00000100:00100000:2.0:1769074843.551230:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2431 00000100:00100000:2.0:1769074843.551237:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625229952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551286:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625229952:12345-0@lo:400:kworker.0 Request processed in 49us (828us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074843.551294:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2431 00000100:00100000:2.0:1769074843.551312:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2432 00000100:00100000:2.0:1769074843.551318:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625230464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551345:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625230464:12345-0@lo:400:kworker.0 Request processed in 27us (855us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074843.551351:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2432 00000100:00100000:2.0:1769074843.551363:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2433 00000100:00100000:2.0:1769074843.551368:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625229824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551393:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625229824:12345-0@lo:400:kworker.0 Request processed in 25us (793us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074843.551399:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2433 00000100:00100000:2.0:1769074843.551410:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2434 00000100:00100000:2.0:1769074843.551416:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625229568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551445:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625229568:12345-0@lo:400:kworker.0 Request processed in 29us (523us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074843.551451:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2434 00000100:00100000:2.0:1769074843.551463:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2435 00000100:00100000:2.0:1769074843.551468:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625229184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074843.551493:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625229184:12345-0@lo:400:kworker.0 Request processed in 25us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.552019:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229440 00000100:00100000:0.0:1769074843.552026:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229440 00000100:00100000:0.0:1769074843.552059:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229312 00000100:00100000:0.0:1769074843.552062:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229312 00000100:00100000:0.0:1769074843.552077:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625230336 00000100:00100000:0.0:1769074843.552079:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625230336 00000100:00100000:0.0:1769074843.552093:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625230208 00000100:00100000:0.0:1769074843.552095:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625230208 00000100:00100000:0.0:1769074843.552144:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1529 00000100:00100000:0.0:1769074843.552151:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625229440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.552191:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625229440:12345-0@lo:400:kworker.0 Request processed in 41us (1776us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.552197:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1529 00000100:00100000:0.0:1769074843.552209:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1530 00000100:00100000:0.0:1769074843.552214:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625229312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.552235:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625229312:12345-0@lo:400:kworker.0 Request processed in 22us (1687us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.552241:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1530 00000100:00100000:0.0:1769074843.552250:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1531 00000100:00100000:0.0:1769074843.552254:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625230336:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.552278:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625230336:12345-0@lo:400:kworker.0 Request processed in 23us (1632us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.552283:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1531 00000100:00100000:0.0:1769074843.552292:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1532 00000100:00100000:0.0:1769074843.552297:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625230208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.552322:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625230208:12345-0@lo:400:kworker.0 Request processed in 24us (1253us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.552327:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1532 00000100:00100000:0.0:1769074843.552363:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2436 00000100:00100000:0.0:1769074843.552371:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625229696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.552400:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625229696:12345-0@lo:400:kworker.0 Request processed in 31us (1366us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.552405:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2436 00000100:00100000:1.0:1769074843.552558:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625228928 00000100:00100000:1.0:1769074843.552564:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625228928 00000100:00100000:1.0:1769074843.552592:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:1.0:1769074843.552598:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649910850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625228928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074843.552635:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649910850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625228928:12345-0@lo:400:kworker.0 Request processed in 37us (2334us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074843.552641:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00080000:1.0:1769074843.552674:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074843.552678:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074843.552681:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074843.552684:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074843.552917:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625229184:0@lo:400:kworker.0 00000100:00100000:1.0:1769074843.552930:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625229696:0@lo:400:kworker.0 00000100:00100000:1.0:1769074843.552939:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625230208:0@lo:400:kworker.0 00000100:00100000:1.0:1769074843.552951:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625229568:0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.553149:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625229056 00000100:00100000:0.0:1769074843.553155:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625229056 00000100:00100000:0.0:1769074843.553194:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625230080 00000100:00100000:0.0:1769074843.553198:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625230080 00000100:00100000:0.0:1769074843.553217:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2711 00000100:00100000:0.0:1769074843.553224:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625229056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.553269:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625229056:12345-0@lo:400:kworker.0 Request processed in 46us (2382us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.553275:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2711 00000100:00100000:0.0:1769074843.553286:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2712 00000100:00100000:0.0:1769074843.553291:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625230080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074843.553321:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625230080:12345-0@lo:400:kworker.0 Request processed in 30us (2367us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074843.553326:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2712 00000100:00100000:2.0:1769074843.553419:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2435 00000100:00100000:3.0:1769074843.553687:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625229312:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.553700:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625229824:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.553711:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625230336:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.553721:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625228928:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.553733:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625229440:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.553743:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625229952:0@lo:400:kworker.0 00000100:00100000:3.0:1769074843.553754:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625230464:0@lo:400:kworker.0 00000100:00100000:1.0:1769074843.553939:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625229056:0@lo:400:kworker.0 00000100:00100000:1.0:1769074843.553951:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625230080:0@lo:400:kworker.0 00010000:00010000:2.0:1769074844.065910:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074844.620985:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074844.621162:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015296 00000100:00100000:3.0:1769074844.621177:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015296 00000100:00100000:3.0:1769074844.621253:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2437 00000100:00100000:3.0:1769074844.621263:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007617015296:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074844.621337:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:2406:x1855007617015296:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 75us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074844.621345:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2437 00000100:00080000:3.0:1769074844.621371:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074844.621604:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074844.621635:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074844.621655:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074844.621688:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074844.621766:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015552 00000100:00100000:3.0:1769074844.621771:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015552 00000100:00100000:3.0:1769074844.621827:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015680 00000100:00100000:3.0:1769074844.621830:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015680 00000100:00100000:3.0:1769074844.621854:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1533 00000100:00100000:3.0:1769074844.621861:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617015552:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074844.621898:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617015552:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 38us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074844.621904:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1533 00000100:00100000:3.0:1769074844.621916:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1534 00000100:00100000:3.0:1769074844.621921:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617015680:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074844.621945:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617015680:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074844.621951:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1534 00000100:00100000:3.0:1769074844.621967:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015168 00000100:00100000:3.0:1769074844.621970:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015168 00000100:00100000:3.0:1769074844.621992:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 371 00000100:00100000:3.0:1769074844.621997:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649911450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617015168:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074844.622020:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649911450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617015168:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 22us (384us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074844.622026:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 371 00000100:00100000:3.0:1769074844.622249:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015424 00000100:00100000:3.0:1769074844.622254:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015424 00000100:00100000:3.0:1769074844.622281:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2438 00000100:00100000:3.0:1769074844.622287:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617015424:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074844.622316:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617015424:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (660us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074844.622321:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2438 00000100:00080000:3.0:1769074844.622335:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074844.622339:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074844.622341:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074844.970578:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074844.972593:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617015808 00000100:00100000:2.0:1769074844.972610:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617015808 00000100:00100000:2.0:1769074844.972709:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13535 00000100:00100000:2.0:1769074844.972721:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66558:x1855007617015808:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:2.0:1769074844.972744:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074844.972772:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd5540f lrc: 2/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c74b expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074844.972860:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 3/1,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074844.972875:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.972888:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.972902:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 3/1,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074844.972909:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.972929:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973013:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074844.973022:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973028:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:2.0:1769074844.973073:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4b:0x0]: returning size 2097152 00000004:00200000:2.0:1769074844.973078:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074844.973114:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973126:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973136:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973142:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973148:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973159:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973165:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074844.973168:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973175:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973188:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b64effa100) 00010000:00010000:2.0:1769074844.973192:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074844.973197:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 3/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973204:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89c00/0x7fe2323adfd55416 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973217:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973223:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 2/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973228:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973233:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074844.973236:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973242:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 4/0,0 mode: PR/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973253:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b64effa900) 00010000:00010000:2.0:1769074844.973255:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074844.973258:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 2/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973264:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd5541d lrc: 0/0,0 mode: --/PR res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074844.973296:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd5540f lrc: 0/0,0 mode: --/CR res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c74b expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074844.973311:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c74b expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074844.973319:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:2.0:1769074844.973321:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074844.973324:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88400, rc 0) 00000100:00100000:2.0:1769074844.973410:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66558:x1855007617015808:12345-192.168.204.60@tcp:101:rm.0 Request processed in 691us (2841us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074844.973418:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13535 00000100:00100000:2.0:1769074844.998382:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074845.010931:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016064 00000100:00100000:3.0:1769074845.010949:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016064 00000100:00100000:3.0:1769074845.011012:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13536 00000100:00100000:3.0:1769074845.011022:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66558:x1855007617016064:12345-192.168.204.60@tcp:36:rm.0 00010000:00010000:3.0:1769074845.011075:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1769074845.011113:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55336 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c721 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.011133:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88400/0x7fe2323adfd55424 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c74b expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.011145:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88c00/0x7fe2323adfd551d1 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c6b1 expref: 7 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.011155:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b681697700) 00010000:00010000:3.0:1769074845.011165:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074845.011191:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 3/0,1 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074845.011205:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 3/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.011214:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 3/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.011226:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,1 mode: --/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074845.011232:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.011238:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.011317:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074845.011324:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.011331:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00000040:00080000:3.0:1769074845.011799:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].66, 64 off13448 00000004:00080000:3.0:1769074845.011853:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1832854045867ns, next at 1833854042001ns, rc = 0 00000040:00080000:3.0:1769074845.011877:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].69, 64 off13640 00000004:00080000:3.0:1769074845.011882:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1832854075511ns, next at 1833854074454ns, rc = 0 00000040:00080000:3.0:1769074845.011891:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].70, 64 off13704 00010000:00010000:3.0:1769074845.012252:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074845.012263:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012270:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012277:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 4/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012283:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 4/0,1 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012288:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012304:0:6514:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1769074845.012321:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012327:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 2/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012333:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012339:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074845.012343:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012349:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 4/0,0 mode: EX/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012358:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b6543ea100) 00010000:00010000:3.0:1769074845.012361:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1769074845.012365:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 2/0,0 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.012370:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb600/0x7fe2323adfd55439 lrc: 0/0,0 mode: --/EX res: [0x240000404:0x4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074845.026150:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074845.026164:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 2/0,0 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074845.026175:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febba00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074845.026180:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 2/0,0 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074845.026189:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074845.026206:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4b:0x0].0x0 (ffff89b6543ea100) 00010000:00010000:2.0:1769074845.026209:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1769074845.026214:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 1/0,0 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074845.026220:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febba00/0x7fe2323adfd55440 lrc: 0/0,0 mode: --/PW res: [0x240000404:0x4b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074845.026240:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074845.026242:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074845.026243:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074845.032794:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 4/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032838:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 4/0,1 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032844:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 3/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032855:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 4/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032860:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074845.032864:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 4/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032871:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 5/0,0 mode: CW/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032889:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0x0 (ffff89b6543ea200) 00010000:00010000:3.0:1769074845.032892:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074845.032897:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 3/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032903:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba800/0x7fe2323adfd5542b lrc: 0/0,0 mode: --/CW res: [0x240000405:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00000004:00010000:3.0:1769074845.032923:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000031927253 reply state 00000000ebc4aef5 transno 4294968366 ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,1 mode: PW/PW res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032935:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.032942:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b6543eaa00) 00010000:00010000:3.0:1769074845.032944:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074845.032947:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074845.033057:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66558:x1855007617016064:12345-192.168.204.60@tcp:36:rm.0 Request processed in 22032us (34680us total) trans 4294968366 rc 0/0 00000100:00100000:3.0:1769074845.033066:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13536 00000001:00080000:0.0:1769074845.033779:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968366 is committed 00000004:00080000:0.0:1769074845.033788:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 67 committed 00000004:00080000:0.0:1769074845.033800:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 64 committed 00000100:00100000:0.0:1769074845.033973:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625230592:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1769074845.034025:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074845.034056:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074845.034063:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625230720:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1769074845.034088:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1769074845.034649:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.034660:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 2/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.034667:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.034672:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074845.034676:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.034683:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 4/0,0 mode: TXN/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.034695:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x7:0x0].0xaa02ef2c (ffff89b6543eaa00) 00010000:00010000:3.0:1769074845.034698:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074845.034702:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 2/0,0 mode: --/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:3.0:1769074845.034707:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febaa00/0x7fe2323adfd55432 lrc: 0/0,0 mode: --/TXN res: [0x240000405:0x7:0x0].0xaa02ef2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00000100:00100000:3.0:1769074845.034737:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625230592 00000100:00100000:3.0:1769074845.034745:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625230592 00000100:00100000:3.0:1769074845.034849:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625230720 00000100:00100000:3.0:1769074845.034853:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625230720 00000100:00100000:3.0:1769074845.034881:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14488 00000100:00100000:3.0:1769074845.034891:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625230592:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1769074845.034928:0:28955:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:53 count 1 00010000:00010000:3.0:1769074845.034996:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035020:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035027:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074845.035029:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074845.035032:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035058:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035072:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 9 pid: 36516 timeout: 1932 lvb_type: 0 00000100:00100000:3.0:1769074845.035084:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625230848:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074845.035161:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074845.035178:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14489 00000100:00100000:3.0:1769074845.035186:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625230720:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:3.0:1769074845.035199:0:8398:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:53 count 1 00010000:00010000:3.0:1769074845.035215:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035226:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035233:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074845.035235:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074845.035237:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 4/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035256:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 9 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.035265:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 9 pid: 36516 timeout: 1932 lvb_type: 0 00000100:00100000:3.0:1769074845.035274:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625230976:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074845.035299:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266600 going to sleep for 11 seconds 00000100:00080000:2.0:1769074845.035480:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074845.035485:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074845.042243:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625230848:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074845.042279:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074845.042311:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625230976:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074845.042323:0:8398:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074845.051063:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074845.053235:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074845.053289:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016192 00000100:00100000:3.0:1769074845.053298:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016192 00010000:00010000:3.0:1769074845.053337:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00000100:00100000:3.0:1769074845.053462:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016320 00000100:00100000:3.0:1769074845.053466:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016320 00010000:00010000:3.0:1769074845.053483:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00000100:00100000:2.0:1769074845.054666:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7210 00000100:00100000:2.0:1769074845.054686:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617016192:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074845.054705:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074845.054714:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00010000:00010000:2.0:1769074845.054731:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00010000:00010000:2.0:1769074845.054762:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febb000/0x7fe2323adfd55313 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c70c expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00010000:00010000:2.0:1769074845.054778:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074845.054780:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074845.054785:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074845.054802:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074845.054976:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7211 00000100:00100000:1.0:1769074845.054987:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617016320:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074845.054998:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074845.055004:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00010000:00010000:1.0:1769074845.055015:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00010000:00010000:1.0:1769074845.055030:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55328 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c71a expref: 10 pid: 36516 timeout: 1932 lvb_type: 0 00010000:00010000:1.0:1769074845.055039:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074845.055041:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074845.055043:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.055050:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.055078:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074845.055149:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617016320:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 165us (1923us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074845.055160:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7211 00010000:00010000:1.0:1769074845.055200:0:8398:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.055207:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.055218:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.055227:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.055244:0:8398:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1769074845.058477:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+8:3649:x1855007625230720:12345-0@lo:6:osp-syn-0-1.0 Request processed in 23282us (24377us total) trans 21474841294 rc 0/0 00000100:00100000:1.0:1769074845.058491:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14489 00010000:00010000:1.0:1769074845.058625:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074845.058636:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.058649:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074845.058653:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.058663:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.058690:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 1/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.058697:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67feba400/0x7fe2323adfd5544e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.058714:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074845.058716:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074845.058718:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074845.059037:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074845.059108:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617016192:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 4419us (8040us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074845.059117:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7210 00000001:00080000:0.0:1769074845.061238:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841294 is committed 00000001:00080000:0.0:1769074845.061256:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841292 is committed 00010000:00010000:3.0:1769074845.063664:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.063685:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.063697:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074845.063704:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1769074845.063869:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2b800/2, rc 0, transno 21474841294 00000100:00100000:2.0:1769074845.063880:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625230720:0@lo:6:osp-syn-0-1.0 00010000:00010000:3.0:1769074845.064006:0:28955:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000001:00080000:2.0:1769074845.071267:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874622 is committed 00000001:00080000:2.0:1769074845.071287:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874619 is committed 00000100:00100000:3.0:1769074845.074394:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625230592:12345-0@lo:6:osp-syn-1-1.0 Request processed in 39490us (40354us total) trans 17179874622 rc 0/0 00000100:00100000:3.0:1769074845.074409:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14488 00010000:00010000:1.0:1769074845.075163:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074845.075175:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.075187:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074845.075191:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.075202:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.075222:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.075229:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55447 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074845.075247:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074845.075249:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074845.075251:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1769074845.079624:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2b100, transno 17179874622 00000100:00100000:0.0:1769074845.079658:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 17179874622 gen 2 00000004:00080000:0.0:1769074845.079662:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f28a80, transno 17179874620 00000004:00080000:0.0:1769074845.079665:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2aa00, transno 17179874621 00000004:00080000:0.0:1769074845.079671:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2b100/2, rc 0, transno 17179874622 00000100:00100000:0.0:1769074845.079681:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625230592:0@lo:6:osp-syn-1-1.0 00000004:00080000:3.0:1769074845.079897:0:9490:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0001: 1 changes, 0 in progress, 0 in flight, 3 done 00010000:00010000:2.0:1769074845.085025:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074845.597064:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625231104:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074845.597149:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074845.597200:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074845.597207:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625231232:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074845.597210:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625231104 00000100:00100000:0.0:1769074845.597218:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625231104 00000100:00100000:2.0:1769074845.597230:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074845.597267:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625231232 00000100:00100000:0.0:1769074845.597270:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625231232 00000100:00100000:2.0:1769074845.597282:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2713 00000100:00100000:2.0:1769074845.597292:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625231104:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074845.597327:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446146048 avail 1299906560 00000100:00100000:2.0:1769074845.597388:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625231104:12345-0@lo:41:osp-pre-1-0.0 Request processed in 98us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074845.598857:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2714 00000100:00100000:0.0:1769074845.598876:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625231232:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074845.598918:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445392384 avail 1299152896 00000100:00100000:0.0:1769074845.598977:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625231232:12345-0@lo:41:osp-pre-0-1.0 Request processed in 109us (1745us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074845.598984:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2714 00000100:00100000:0.0:1769074845.599064:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625231232:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074845.599079:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625231104:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074845.601612:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2713 00000100:00100000:2.0:1769074846.109044:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074846.109065:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074846.109067:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625231488:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074846.109074:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625231616:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1769074846.111128:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074846.115745:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074846.118758:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625231616 00000100:00100000:3.0:1769074846.118774:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625231616 00000100:00100000:3.0:1769074846.118884:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1224 00000100:00100000:3.0:1769074846.118896:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625231616:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074846.118933:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074846.119032:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625231616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 140us (3296us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074846.119040:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1224 00000100:00080000:0.0:1769074846.122402:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074846.122626:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625231616:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074846.125710:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074846.125773:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625231360:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074846.125804:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625231488 00000100:00100000:2.0:1769074846.125823:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074846.125827:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625231488 00000100:00100000:1.0:1769074846.128404:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625231360 00000100:00100000:1.0:1769074846.128419:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625231360 00000100:00100000:1.0:1769074846.128650:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1225 00000100:00100000:1.0:1769074846.128659:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625231488:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074846.128685:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074846.128763:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625231488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 105us (3061us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074846.128769:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1225 00000100:00100000:1.0:1769074846.128786:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1226 00000100:00100000:1.0:1769074846.128792:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625231360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074846.128827:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 1073152 granted 23426368 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074846.128857:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625231360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (3032us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074846.128864:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1226 00000100:00080000:0.0:1769074846.128944:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074846.129609:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 21474841294 gen 2 00000004:00080000:2.0:1769074846.129617:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f29c00, transno 21474841291 00000004:00080000:2.0:1769074846.129634:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5444a6a00, transno 21474841293 00000004:00080000:2.0:1769074846.129637:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2b800, transno 21474841294 00000100:00100000:2.0:1769074846.129653:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625231488:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074846.129673:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625231360:0@lo:13:osp-pre-1-0.0 00000004:00080000:0.0:1769074846.129889:0:9035:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 3 done 00000004:00080000:0.0:1769074846.623264:0:9492:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1834465454157ns, next at 1835465451293ns, rc = 0 00000100:00100000:0.0:1769074846.623293:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074846.623315:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625231744:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074846.623387:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074846.623438:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625231744 00000100:00100000:0.0:1769074846.623446:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625231744 00000100:00100000:0.0:1769074846.623517:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1227 00000100:00100000:0.0:1769074846.623526:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625231744:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074846.623549:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 1073152 granted 23426368 pending 0 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074846.623594:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625231744:12345-0@lo:13:osp-pre-1-1.0 Request processed in 69us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074846.623601:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1227 00000100:00100000:0.0:1769074846.623648:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625231744:0@lo:13:osp-pre-1-1.0 00000001:02000400:2.0:1769074846.835453:0:37801:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 204b: FLR write/stale/resync tests with self-extending primary ========================================================== 04:40:44 (1769074844) 00000100:00100000:2.0:1769074847.002613:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.002709:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016448 00000100:00100000:0.0:1769074847.002719:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016448 00000100:00100000:0.0:1769074847.002797:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13537 00000100:00100000:0.0:1769074847.002829:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66719:x1855007617016448:12345-192.168.204.60@tcp:41:stat.0 00000004:00000024:0.0:1769074847.002890:0:6514:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7830949888 avail 7366832128 00000100:00100000:0.0:1769074847.002968:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66719:x1855007617016448:12345-192.168.204.60@tcp:41:stat.0 Request processed in 162us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.002977:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13537 00000100:00080000:2.0:1769074847.003642:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074847.049212:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.049247:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016576 00000100:00100000:3.0:1769074847.049257:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016576 00000100:00100000:3.0:1769074847.049386:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13538 00000100:00100000:3.0:1769074847.049398:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66723:x1855007617016576:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.049504:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.049524:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55455 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c752 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.049591:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049606:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049619:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049662:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049670:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049676:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049724:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049732:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049738:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049743:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049749:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074847.049752:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049760:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049771:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074847.049774:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074847.049779:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049785:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5545c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049798:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049822:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049828:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049834:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074847.049836:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049842:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049848:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xebf837e7 (ffff89b6543eaa00) 00010000:00010000:3.0:1769074847.049850:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074847.049853:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049857:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55463 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.049888:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55455 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c752 expref: 15 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.049897:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074847.049899:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074847.049903:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55455 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c752 expref: 15 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.049912:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba800, rc 0) 00000100:00100000:3.0:1769074847.049966:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66723:x1855007617016576:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 570us (765us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074847.049973:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13538 00000100:00100000:0.0:1769074847.056538:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.056575:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016832 00000100:00100000:0.0:1769074847.056585:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016832 00000100:00100000:0.0:1769074847.056638:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13539 00000100:00100000:0.0:1769074847.056648:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66723:x1855007617016832:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:0.0:1769074847.056717:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074847.056750:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd550ff lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c66b expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.056769:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074847.056775:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074847.056867:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.056882:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.056893:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.056908:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.056914:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.056920:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000040:00080000:0.0:1769074847.058366:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].85, 2168 off90640 00000040:00080000:0.0:1769074847.060547:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000405:0x2:0x0].7, 2368 off47360 00000004:00080000:0.0:1769074847.060563:0:6514:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b64ef5b180 with version 18 00000004:00080000:0.0:1769074847.061139:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 18 rpc_version 18 00000020:00100000:0.0:1769074847.061184:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1769074847.061209:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b54b426a00 x1855007625232000/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1769074847.061235:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625232000:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1769074847.061305:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074847.061334:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7f300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074847.061623:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625232000 00000100:00100000:0.0:1769074847.061631:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625232000 00000100:00100000:0.0:1769074847.061674:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2715 00000100:00100000:0.0:1769074847.061682:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625232000:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1769074847.063741:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625232000:12345-0@lo:1000:osp_up1-0.0 Request processed in 2051us (2427us total) trans 4294968367 rc 0/0 00000100:00100000:0.0:1769074847.063754:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2715 00000001:00080000:3.0:1769074847.065163:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944098 is committed 00000020:00080000:3.0:1769074847.065177:0:32377:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 4294969366 batchid updated 00000020:00080000:3.0:1769074847.065193:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969366 00000100:00100000:0.0:1769074847.065291:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625232000:0@lo:1000:osp_up1-0.0 00000004:00080000:3.0:1769074847.065296:0:32406:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b67fc18d80 with version 19 00000004:00080000:0.0:1769074847.065307:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 19 rpc_version 19 00000020:00080000:3.0:1769074847.065313:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294969366 cancel update log [0x240000405:0x2:0x0].6: rc = 0 00000020:00100000:0.0:1769074847.065339:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1769074847.065348:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b54b424a80 x1855007625232256/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1769074847.065371:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b424a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625232256:0@lo:1000:osp_up1-0.0 00000020:00080000:3.0:1769074847.065395:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294969366 cancel update log [0x200000bd0:0x2:0x0].84: rc = 0 00000100:00100000:0.0:1769074847.065446:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074847.065473:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7f300 going to sleep for 11 seconds 00000100:00100000:0.0:1769074847.065489:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625232256 00000100:00100000:0.0:1769074847.065495:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625232256 00000100:00100000:0.0:1769074847.065535:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2716 00000100:00100000:0.0:1769074847.065543:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625232256:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1769074847.065661:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:32392:x1855007625232256:12345-0@lo:1000:osp_up1-0.0 Request processed in 117us (213us total) trans 4294968368 rc 0/0 00000020:00080000:3.0:1769074847.065663:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969366 00000100:00100000:0.0:1769074847.065672:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2716 00000100:00100000:0.0:1769074847.066010:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b424a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625232256:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1769074847.066544:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x206:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066561:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066572:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066591:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066597:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066603:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066633:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 3/0,1 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066639:0:6514:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 3/0,1 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.066649:0:6514:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 3/0,1 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074847.066663:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6514:1855007625232384:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1769074847.066704:0:6514:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074847.066737:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6480f3180 going to sleep for 11 seconds 00000100:00100000:0.0:1769074847.066795:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625232384 00000100:00100000:0.0:1769074847.066801:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625232384 00000100:00100000:0.0:1769074847.066991:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2717 00000100:00100000:0.0:1769074847.067000:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6514:x1855007625232384:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1769074847.067015:0:7791:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.067033:0:7791:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 2/0,0 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7fe2323adfd55486 expref: -99 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.067052:0:7791:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7fe2323adfd55486 expref: 8 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.067077:0:7791:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd55486 expref: 8 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.067086:0:7791:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x8:0x0].0x0 (ffff89b643a80300) 00010000:00010000:0.0:1769074847.067089:0:7791:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.067092:0:7791:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134a00, rc 0) 00000100:00100000:0.0:1769074847.067130:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6514:x1855007625232384:12345-0@lo:101:mdt00_002.0 Request processed in 131us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.067137:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2717 00000100:00100000:2.0:1769074847.067201:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6514:1855007625232384:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1769074847.067215:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.067223:0:6514:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1769074847.067242:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008269c061 reply state 000000000168708d transno 8589944098 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067252:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067258:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effae00) 00010000:00010000:2.0:1769074847.067260:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074847.067263:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074847.067268:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.067275:0:6514:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1769074847.067284:0:6514:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000405:0x8:0x0] 00000004:00010000:2.0:1769074847.067289:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008269c061 reply state 000000000168708d transno 8589944098 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067298:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067303:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x206:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074847.067305:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074847.067306:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1769074847.067311:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x206:0x0]: returning size 4096 00010000:00010000:2.0:1769074847.067319:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067325:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067330:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067337:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067343:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.067346:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067353:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067361:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074847.067362:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.067365:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067369:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd5546a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000004:00010000:2.0:1769074847.067379:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008269c061 reply state 000000000168708d transno 8589944098 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067384:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.067387:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xebf837e7 (ffff89b643a80200) 00010000:00010000:2.0:1769074847.067388:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074847.067389:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 2 00010000:00080000:2.0:1769074847.067521:0:6514:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1769074847.067540:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66723:x1855007617016832:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 10892us (11011us total) trans 8589944098 rc 0/0 00000100:00100000:2.0:1769074847.067547:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13539 00010000:00010000:0.0:1769074847.067605:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067613:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067618:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067623:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.067626:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067632:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067641:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xebf837e7 (ffff89b643a80200) 00010000:00010000:0.0:1769074847.067643:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.067646:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067651:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55471 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067665:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067671:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067677:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067682:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.067685:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067690:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067696:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x206:0x0].0x0 (ffff89b64effa200) 00010000:00010000:0.0:1769074847.067698:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.067701:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067706:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55478 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x206:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067717:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067722:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067728:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067733:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.067735:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067741:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067748:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effae00) 00010000:00010000:0.0:1769074847.067750:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.067753:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.067758:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5547f lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074847.119702:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.120005:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617016960 00000100:00100000:0.0:1769074847.120016:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617016960 00000100:00100000:0.0:1769074847.120091:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13540 00000100:00100000:0.0:1769074847.120100:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:66726:x1855007617016960:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.120124:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.120145:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55494 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c759 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.120207:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd5549b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.120222:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd5549b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.120234:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd5549b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.120281:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1769074847.120284:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074847.120314:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd5549b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.120339:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55494 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c759 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.120358:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd5549b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c759 expref: 14 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.120368:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074847.120371:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769074847.120375:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135200, rc 0) 00000100:00100000:0.0:1769074847.120449:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66726:x1855007617016960:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 349us (755us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.120457:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13540 00000100:00100000:1.0:1769074847.127861:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.128458:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017088 00000100:00100000:0.0:1769074847.128469:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017088 00000100:00100000:0.0:1769074847.128531:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13541 00000100:00100000:0.0:1769074847.128540:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:66726:x1855007617017088:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.128559:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.128579:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd554a2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c760 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.128646:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128661:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128671:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128686:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128693:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128699:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128780:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135a00/0x7fe2323adfd554b7 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x206:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128788:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135a00/0x7fe2323adfd554b7 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128795:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135a00/0x7fe2323adfd554b7 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.128896:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x206:0x0]: returning size 4096 00000004:00200000:0.0:1769074847.128902:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1769074847.128920:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135a00/0x7fe2323adfd554b7 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128932:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128939:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128944:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128951:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128957:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.128960:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128967:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128980:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074847.128983:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.128988:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.128994:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd554a9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129007:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129012:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129018:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129023:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.129026:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129031:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129038:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xebf837e7 (ffff89b643a80200) 00010000:00010000:0.0:1769074847.129041:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.129043:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129049:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134800/0x7fe2323adfd554b0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xebf837e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.129092:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd554a2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c760 expref: 16 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.129105:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135a00/0x7fe2323adfd554b7 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c760 expref: 15 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.129111:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x206:0x0].0x0 (ffff89b643a80800) 00010000:00010000:0.0:1769074847.129114:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1769074847.129116:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135a00, rc 0) 00000100:00100000:0.0:1769074847.129183:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:66726:x1855007617017088:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 644us (1333us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.129191:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13541 00010000:00010000:1.0:1769074847.133783:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074847.134748:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.134973:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017216 00000100:00100000:3.0:1769074847.134982:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017216 00000100:00100000:3.0:1769074847.135035:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13542 00000100:00100000:3.0:1769074847.135044:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+15:66726:x1855007617017216:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.135073:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.135096:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd554be lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c767 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.135155:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd554c5 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.135170:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd554c5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.135181:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd554c5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074847.135226:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd0:0x8:0x0]: returning size 4096 00000004:00200000:3.0:1769074847.135232:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1769074847.135266:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd554be lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c767 expref: 17 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.135281:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd554c5 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c767 expref: 16 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.135296:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:3.0:1769074847.135310:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1769074847.135313:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074847.135375:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:66726:x1855007617017216:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 331us (632us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.135382:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13542 00000100:00100000:2.0:1769074847.140538:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.141128:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017344 00000100:00100000:0.0:1769074847.141140:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017344 00000100:00080000:2.0:1769074847.141200:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074847.141220:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13543 00000100:00100000:0.0:1769074847.141231:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+5:66726:x1855007617017344:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.141256:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.141276:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd554cc lrc: 2/0,0 mode: --/CR res: [0x240000405:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c76e expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.141339:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.141352:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 2/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd55486 expref: 7 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.141361:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 4/1,0 mode: --/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074847.141393:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7fe2323adfd55486 expref: 7 pid: 7791 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074847.141421:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6514:1855007625232512:0@lo:104: 00000100:00100000:0.0:1769074847.141474:0:6514:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074847.141518:0:6496:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625232512 00000100:00100000:0.0:1769074847.141520:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7f180 going to sleep for 11 seconds 00000100:00100000:3.0:1769074847.141525:0:6496:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625232512 00000100:00100000:3.0:1769074847.141623:0:6496:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:3.0:1769074847.141631:0:6496:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6514:x1855007625232512:12345-0@lo:104: 00010000:00010000:3.0:1769074847.141642:0:6496:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.141701:0:6496:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1769074847.141707:0:6496:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6514:x1855007625232512:12345-0@lo:104: Request processed in 78us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.141714:0:6496:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00010000:00010000:2.0:1769074847.141764:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074847.141772:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.141781:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc1908290) 00010000:00010000:2.0:1769074847.141787:0:6501:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074847.141793:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6514:1855007625232512:0@lo:104: 00000004:00010000:2.0:1769074847.141799:0:6501:0:(mdt_handler.c:3944:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.141849:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074847.141861:0:6501:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 2/0,0 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.141881:0:6501:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1769074847.141906:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 1/0,0 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.141913:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b88e00/0x7fe2323adfd55486 lrc: 0/0,0 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd5548d expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074847.141916:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625232640:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1769074847.141936:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074847.141939:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074847.141941:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074847.143164:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074847.143290:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625232640 00000100:00100000:0.0:1769074847.143297:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625232640 00010000:00010000:0.0:1769074847.143335:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7fe2323adfd55486 expref: 8 pid: 7791 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074847.143400:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7212 00000100:00100000:0.0:1769074847.143417:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625232640:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1769074847.143428:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1769074847.145524:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968368 is committed 00000004:00080000:1.0:1769074847.145540:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 68 committed 00000100:00100000:2.0:1769074847.145760:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625232768:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1769074847.145844:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074847.145940:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625232768 00000100:00100000:3.0:1769074847.145947:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625232768 00000100:00100000:3.0:1769074847.146015:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14490 00000100:00100000:3.0:1769074847.146025:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625232768:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1769074847.146052:0:28955:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:54 count 1 00010000:00010000:3.0:1769074847.146119:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.146141:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 7 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.146149:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074847.146151:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074847.146154:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.146180:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 7 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.146193:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 7 pid: 28519 timeout: 1934 lvb_type: 0 00000100:00100000:3.0:1769074847.146204:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625232896:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074847.146266:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc18840 going to sleep for 11 seconds 00000100:00080000:2.0:1769074847.146664:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074847.146962:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd5548d lrc: 0/0,0 mode: --/PW res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7fe2323adfd55486 expref: 6 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.146981:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x8:0x0].0x0 (ffff89b643a80300) 00010000:00010000:0.0:1769074847.146985:0:6498:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/1,0 mode: --/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074847.146994:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074847.147000:0:6498:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 4/1,0 mode: PR/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074847.147208:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074847.147330:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625232640:12345-0@lo:103:ldlm_bl.0 Request processed in 3912us (4168us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.147339:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7212 00010000:00010000:0.0:1769074847.147530:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074847.147539:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00000004:00200000:0.0:1769074847.147599:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000405:0x8:0x0]: returning size 4096 00000004:00200000:0.0:1769074847.147604:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1769074847.147646:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd554cc lrc: 0/0,0 mode: --/CR res: [0x240000405:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c76e expref: 7 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.147662:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134800/0x7fe2323adfd554d3 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c76e expref: 6 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.147670:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x8:0x0].0x0 (ffff89b643a80300) 00010000:00010000:0.0:1769074847.147673:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1769074847.147676:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134800, rc 0) 00000100:00100000:0.0:1769074847.147769:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:66726:x1855007617017344:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 6540us (7240us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.147777:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13543 00000100:00100000:3.0:1769074847.147958:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625232640:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1769074847.148990:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.149155:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017472 00000100:00100000:0.0:1769074847.149163:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017472 00010000:00010000:0.0:1769074847.149209:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 8 pid: 28519 timeout: 1934 lvb_type: 0 00000100:00100000:0.0:1769074847.149245:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7213 00000100:00100000:0.0:1769074847.149254:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617017472:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074847.149268:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074847.149273:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 8 pid: 28519 timeout: 1934 lvb_type: 0 00010000:00010000:0.0:1769074847.149286:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 8 pid: 28519 timeout: 1934 lvb_type: 0 00010000:00010000:0.0:1769074847.149318:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.149320:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.149325:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.149336:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.149354:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074847.149398:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617017472:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 144us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.149419:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7213 00000100:00080000:2.0:1769074847.149653:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1769074847.149863:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd5531a lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c713 expref: 7 pid: 28519 timeout: 1934 lvb_type: 0 00000100:00100000:3.0:1769074847.149879:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625232896:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074847.149896:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.149906:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.149913:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.149923:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.149931:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.149994:0:28955:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1769074847.151479:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3650:x1855007625232768:12345-0@lo:6:osp-syn-1-1.0 Request processed in 5449us (5629us total) trans 17179874623 rc 0/0 00000100:00100000:3.0:1769074847.151490:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14490 00000004:00080000:2.0:1769074847.151917:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f2b800/2, rc 0, transno 17179874623 00000100:00100000:2.0:1769074847.151925:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625232768:0@lo:6:osp-syn-1-1.0 00010000:00010000:1.0:1769074847.152654:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074847.152666:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.152679:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074847.152683:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.152694:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.152714:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.152722:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67feba800/0x7fe2323adfd554da lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.152744:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074847.152746:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074847.152748:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074847.153140:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.153381:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017600 00000100:00100000:2.0:1769074847.153390:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017600 00000100:00100000:2.0:1769074847.153561:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13544 00000100:00100000:2.0:1769074847.153571:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:66726:x1855007617017600:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.153590:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.153609:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd554e1 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c775 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.153672:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153686:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153696:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153732:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153739:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153745:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153848:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153857:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153863:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153869:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153875:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.153878:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153886:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153896:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:2.0:1769074847.153899:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.153903:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153909:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd554e8 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153920:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153925:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153930:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153934:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.153936:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153941:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153948:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa400) 00010000:00010000:2.0:1769074847.153950:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.153953:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153958:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd554ef lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.153991:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd554e1 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c775 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.154000:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:2.0:1769074847.154002:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.154184:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd554e1 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c775 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.154197:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074847.154316:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:66726:x1855007617017600:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 745us (1177us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074847.154324:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13544 00000100:00100000:0.0:1769074847.159004:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.159104:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017728 00000100:00100000:2.0:1769074847.159114:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017728 00000100:00100000:2.0:1769074847.159217:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13545 00000100:00100000:2.0:1769074847.159228:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:66726:x1855007617017728:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.159250:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.159270:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd554f6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c77c expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.159338:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd554fd lrc: 3/1,0 mode: --/PR res: [0x200000405:0x206:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.159352:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd554fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.159365:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd554fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074847.159464:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x206:0x0]: returning size 4096 00000004:00200000:2.0:1769074847.159469:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1769074847.159487:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd554fd lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.159513:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd554f6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c77c expref: 18 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.159530:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89000/0x7fe2323adfd554fd lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x206:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c77c expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.159539:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x206:0x0].0x0 (ffff89b643a80800) 00010000:00010000:2.0:1769074847.159542:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074847.159546:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89000, rc 0) 00000100:00100000:2.0:1769074847.159605:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:66726:x1855007617017728:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 380us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.159613:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13545 00000100:00100000:0.0:1769074847.163625:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.163712:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617017984 00000100:00100000:2.0:1769074847.163722:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617017984 00000100:00100000:2.0:1769074847.163847:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13546 00000100:00100000:2.0:1769074847.163858:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:66726:x1855007617017984:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.163879:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.163882:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074847.163906:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd554c5 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c767 expref: 17 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.163924:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:2.0:1769074847.163936:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074847.163951:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55504 lrc: 2/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c783 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.164069:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 3/0,1 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164083:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164093:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164105:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164111:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164118:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164512:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164523:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164531:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074847.164541:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 0 00010000:00010000:2.0:1769074847.164567:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164575:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164582:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 3/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164589:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164596:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.164600:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164608:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 5/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164617:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.164620:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.164623:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 3/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164629:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd5550b lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000004:00010000:2.0:1769074847.164642:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000881c4d0d reply state 00000000898a444d transno 8589944099 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164652:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.164658:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa000) 00010000:00010000:2.0:1769074847.164660:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074847.164663:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074847.164686:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55504 lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.164708:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 18 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.164716:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074847.164718:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074847.164721:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89200, rc 0) 00000100:00100000:2.0:1769074847.164777:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66726:x1855007617017984:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 920us (1162us total) trans 8589944099 rc 0/0 00000100:00100000:2.0:1769074847.164785:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13546 00000100:00100000:0.0:1769074847.168258:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.168498:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617018240 00000100:00100000:3.0:1769074847.168508:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617018240 00000100:00100000:3.0:1769074847.168569:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13547 00000100:00100000:3.0:1769074847.168579:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66726:x1855007617018240:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.168608:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.168631:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55527 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c78a expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.168746:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.168761:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.168770:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 4/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.168802:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.168851:0:10283:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 20 pid: 6514 timeout: 1935 lvb_type: 0 00000100:00100000:3.0:1769074847.168864:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625233024:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074847.168944:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67fc18840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074847.175085:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.175367:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10283:1855007625233024:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074847.175396:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000100:00100000:1.0:1769074847.176350:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617018368 00000100:00100000:1.0:1769074847.176361:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617018368 00010000:00010000:1.0:1769074847.176412:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 21 pid: 6514 timeout: 1935 lvb_type: 0 00000100:00100000:1.0:1769074847.176444:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7214 00000100:00100000:1.0:1769074847.176453:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2409:x1855007617018368:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074847.176470:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074847.176478:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 21 pid: 6514 timeout: 1935 lvb_type: 0 00010000:00010000:1.0:1769074847.176492:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 21 pid: 6514 timeout: 1935 lvb_type: 0 00010000:00010000:1.0:1769074847.176514:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55519 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c783 expref: 21 pid: 6514 timeout: 1935 lvb_type: 0 00010000:00010000:1.0:1769074847.176531:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa800) 00010000:00010000:1.0:1769074847.176537:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074847.176546:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074847.176554:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074847.176585:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074847.176644:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2409:x1855007617018368:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 191us (1570us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074847.176652:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7214 00010000:00010000:3.0:1769074847.177486:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177497:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074847.177520:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 0 00000004:00080000:3.0:1769074847.177702:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c7:0x0], index 0: 0 00000004:00080000:3.0:1769074847.177709:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2c9:0x0], index 1: 0 00010000:00010000:3.0:1769074847.177842:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177850:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177857:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177863:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074847.177867:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177875:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177888:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa800) 00010000:00010000:3.0:1769074847.177890:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074847.177893:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 2/0,0 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177899:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5552e lrc: 0/0,0 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.177946:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55527 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c78a expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.177955:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa800) 00010000:00010000:3.0:1769074847.177958:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074847.177961:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55527 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c78a expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.177974:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febaa00, rc 0) 00000100:00100000:3.0:1769074847.178035:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66726:x1855007617018240:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9458us (9788us total) trans 8589944100 rc 301/301 00000100:00100000:3.0:1769074847.178043:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13547 00000100:00100000:0.0:1769074847.184134:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.184339:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617018624 00000100:00100000:1.0:1769074847.184349:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617018624 00000100:00100000:1.0:1769074847.184416:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4736 00000100:00100000:1.0:1769074847.184427:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66726:x1855007617018624:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074847.184704:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66726:x1855007617018624:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 277us (581us total) trans 8589944101 rc 0/0 00000100:00100000:1.0:1769074847.184712:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4736 00000100:00100000:1.0:1769074847.187684:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.187765:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617018880 00000100:00100000:2.0:1769074847.187775:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617018880 00000100:00100000:2.0:1769074847.187943:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4737 00000100:00100000:2.0:1769074847.187953:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66726:x1855007617018880:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074847.188189:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66726:x1855007617018880:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 236us (514us total) trans 8589944102 rc 0/0 00000100:00100000:2.0:1769074847.188197:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4737 00000100:00100000:0.0:1769074847.211250:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.211339:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617019136 00000100:00100000:3.0:1769074847.211436:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617019136 00000100:00100000:3.0:1769074847.211500:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13548 00000100:00100000:3.0:1769074847.211509:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66727:x1855007617019136:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074847.211534:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.211558:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5553c lrc: 2/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c791 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.211758:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211773:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211785:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211796:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211820:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211827:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211893:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55551 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211901:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55551 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.211908:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55551 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074847.211919:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 0 00010000:00010000:0.0:1769074847.211999:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212480:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212487:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212495:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212501:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.212506:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212514:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212531:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543eac00) 00010000:00010000:0.0:1769074847.212535:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.212540:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212546:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55543 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212560:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212566:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212572:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212577:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.212580:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212585:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212592:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa000) 00010000:00010000:0.0:1769074847.212594:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.212597:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212603:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5554a lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074847.212641:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5553c lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c791 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.212660:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55551 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c791 expref: 23 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.212668:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b643a80200) 00010000:00010000:0.0:1769074847.212671:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074847.212674:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134e00, rc 0) 00000100:00100000:0.0:1769074847.212730:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66727:x1855007617019136:12345-192.168.204.60@tcp:101:dd.0 Request processed in 1222us (1489us total) trans 8589944103 rc 0/0 00000100:00100000:0.0:1769074847.212739:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13548 00000100:00100000:0.0:1769074847.232333:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.232616:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617019392 00000100:00100000:2.0:1769074847.232628:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617019392 00000100:00100000:2.0:1769074847.232729:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13549 00000100:00100000:2.0:1769074847.232740:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66727:x1855007617019392:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074847.232770:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.232773:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074847.232837:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55551 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c791 expref: 23 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.232862:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b643a80200) 00010000:00010000:2.0:1769074847.232872:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074847.232887:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5555f lrc: 2/0,0 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c798 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.232936:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55566 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.232948:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55566 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.232957:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55566 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00080000:2.0:1769074847.233185:0:10283:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c8:0x0], index 0: 0 00000040:00080000:2.0:1769074847.233289:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2463, 88 off210456 00000004:00080000:2.0:1769074847.233315:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1835075507929ns, next at 1809352413082ns, rc = 0 00000040:00080000:2.0:1769074847.233348:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2464, 88 off210544 00000004:00080000:2.0:1769074847.233353:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1835075545687ns, next at 1809352413082ns, rc = 0 00000040:00080000:2.0:1769074847.233376:0:10283:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2468, 88 off211136 00000004:00080000:2.0:1769074847.233382:0:10283:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1835075575110ns, next at 1819234810192ns, rc = 0 00000001:00080000:3.0:1769074847.238153:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944104 is committed 00000004:00080000:3.0:1769074847.238167:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1964 committed 00000004:00080000:3.0:1769074847.238183:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1962 committed 00000004:00080000:3.0:1769074847.238198:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1961 committed 00000100:00100000:2.0:1769074847.238323:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625233280:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074847.238376:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625233152:0@lo:2:osp-syn-0-0.0 00010000:00010000:0.0:1769074847.238436:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.238446:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.238454:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.238460:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.238465:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.238472:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.238489:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa000) 00010000:00010000:0.0:1769074847.238493:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.238498:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.238505:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55512 lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074847.238566:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074847.238674:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074847.238721:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625233280 00000100:00100000:0.0:1769074847.238729:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625233280 00000100:00100000:0.0:1769074847.238778:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625233152 00000100:00100000:0.0:1769074847.238782:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625233152 00000100:00100000:0.0:1769074847.238803:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14491 00000100:00100000:0.0:1769074847.238839:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625233280:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074847.238912:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14492 00000100:00100000:3.0:1769074847.238922:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625233152:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074847.239034:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625233152:12345-0@lo:2:osp-syn-0-0.0 Request processed in 114us (359us total) trans 21474841296 rc 0/0 00000100:00100000:3.0:1769074847.239041:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14492 00010000:00010000:2.0:1769074847.239202:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55566 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c798 expref: 18 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.239213:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.239216:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:2.0:1769074847.239230:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5555f lrc: 0/0,0 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c798 expref: 18 pid: 10283 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1769074847.239245:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55566 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0xa/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c798 expref: 17 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.239276:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.239279:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:2.0:1769074847.239281:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074847.239346:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:66727:x1855007617019392:12345-192.168.204.60@tcp:101:dd.0 Request processed in 6608us (7023us total) trans 8589944104 rc 0/0 00000100:00100000:2.0:1769074847.239353:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13549 00000100:00100000:0.0:1769074847.239575:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625233280:12345-0@lo:2:osp-syn-0-0.0 Request processed in 737us (1174us total) trans 21474841295 rc 0/0 00000100:00100000:0.0:1769074847.239582:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14491 00000004:00080000:0.0:1769074847.239627:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef5180/2, rc 0, transno 21474841296 00000100:00100000:0.0:1769074847.239632:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625233152:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074847.239793:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547ef4e00/2, rc 0, transno 21474841295 00000100:00100000:2.0:1769074847.239798:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625233280:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074847.239893:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074847.239905:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625233408:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074847.239936:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074847.240013:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625233408 00000100:00100000:1.0:1769074847.240020:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625233408 00000100:00100000:1.0:1769074847.240055:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14493 00000100:00100000:1.0:1769074847.240081:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625233408:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074847.240252:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625233408:12345-0@lo:2:osp-syn-1-0.0 Request processed in 173us (315us total) trans 17179874624 rc 0/0 00000100:00100000:1.0:1769074847.240259:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14493 00000004:00080000:2.0:1769074847.240612:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54c844000/2, rc 0, transno 17179874624 00000100:00100000:2.0:1769074847.240620:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625233408:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074847.245682:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.245736:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617019520 00000100:00100000:2.0:1769074847.245747:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617019520 00000100:00100000:2.0:1769074847.245921:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13550 00000100:00100000:2.0:1769074847.245933:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:66727:x1855007617019520:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:2.0:1769074847.245958:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.245980:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5556d lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c79f expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.246027:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55574 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.246041:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55574 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.246053:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55574 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.246181:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5556d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c79f expref: 19 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.246201:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55574 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c79f expref: 18 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.246210:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.246213:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1769074847.246217:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074847.246356:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:66727:x1855007617019520:12345-192.168.204.60@tcp:101:dd.0 Request processed in 423us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.246365:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13550 00000100:00100000:3.0:1769074847.262573:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.262614:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.262847:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617019776 00000100:00100000:0.0:1769074847.262857:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617019776 00000100:00100000:0.0:1769074847.262931:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617019648 00000100:00100000:0.0:1769074847.262934:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617019648 00000100:00100000:0.0:1769074847.262970:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8749 00000100:00100000:0.0:1769074847.262977:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617019776:12345-192.168.204.60@tcp:4:dd.0 00002000:00010000:0.0:1769074847.263044:0:8404:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1769074847.263053:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.263075:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.263077:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.263082:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.263097:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00080000:1.0:1769074847.264150:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074847.264195:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8750 00000100:00100000:1.0:1769074847.264209:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617019648:12345-192.168.204.60@tcp:4:dd.0 00002000:00010000:1.0:1769074847.264245:0:16436:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c7:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074847.264253:0:16436:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.264264:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.264266:0:16436:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.264270:0:16436:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.264283:0:16436:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1769074847.288998:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841297 is committed 00000001:00080000:0.0:1769074847.289025:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841296 is committed 00002000:00010000:3.0:1769074847.289251:0:16436:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1769074847.289256:0:16436:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb mtime from disk: 0 -> 1769074846 00002000:00010000:3.0:1769074847.289258:0:16436:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb ctime from disk: 0 -> 1769074846 00002000:00010000:3.0:1769074847.289260:0:16436:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:3.0:1769074847.289274:0:16436:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.289286:0:16436:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.289295:0:16436:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.289303:0:16436:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074847.289307:0:16436:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.289316:0:16436:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.289336:0:16436:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.289344:0:16436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55582 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16436 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074847.291119:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617019648:12345-192.168.204.60@tcp:4:dd.0 Request processed in 26900us (28490us total) trans 21474841297 rc 0/0 00000100:00100000:2.0:1769074847.291134:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8750 00000001:00080000:0.0:1769074847.303100:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841298 is committed 00002000:00010000:3.0:1769074847.304657:0:8404:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:3.0:1769074847.304665:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb mtime from disk: 0 -> 1769074846 00002000:00010000:3.0:1769074847.304669:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb ctime from disk: 0 -> 1769074846 00002000:00010000:3.0:1769074847.304671:0:8404:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:3.0:1769074847.304685:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.304698:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.304708:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.304717:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074847.304721:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.304730:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.304751:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.304760:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5557b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074847.304913:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617019776:12345-192.168.204.60@tcp:4:dd.0 Request processed in 41930us (42345us total) trans 21474841298 rc 0/0 00000100:00100000:3.0:1769074847.304924:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8749 00000100:00100000:1.0:1769074847.312096:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.312265:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617020032 00000100:00100000:2.0:1769074847.312276:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617020032 00000100:00100000:2.0:1769074847.312576:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4738 00000100:00100000:2.0:1769074847.312588:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:66727:x1855007617020032:12345-192.168.204.60@tcp:35:dd.0 00000100:00100000:2.0:1769074847.313077:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66727:x1855007617020032:12345-192.168.204.60@tcp:35:dd.0 Request processed in 488us (1014us total) trans 8589944105 rc 0/0 00000100:00100000:2.0:1769074847.313086:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4738 00000100:00100000:0.0:1769074847.347307:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.347663:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617020288 00000100:00100000:2.0:1769074847.347677:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617020288 00000100:00100000:2.0:1769074847.347745:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13551 00000100:00100000:2.0:1769074847.347754:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66728:x1855007617020288:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.347779:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.347825:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55589 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7a6 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.347985:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348000:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348113:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348127:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348134:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348140:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348216:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348224:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348229:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348239:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348244:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.348248:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348255:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348268:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074847.348270:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.348275:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348281:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55590 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348369:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348377:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348383:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348388:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.348391:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348613:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348629:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effac00) 00010000:00010000:2.0:1769074847.348632:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.348636:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348642:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd55597 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.348678:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55589 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7a6 expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.348688:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074847.348691:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074847.348695:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55589 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7a6 expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.348706:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074847.348867:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66728:x1855007617020288:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1113us (1577us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074847.348886:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13551 00000100:00100000:1.0:1769074847.361053:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.361358:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617020544 00000100:00100000:2.0:1769074847.361372:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617020544 00000100:00100000:2.0:1769074847.361445:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13552 00000100:00100000:2.0:1769074847.361457:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66728:x1855007617020544:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.361480:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.361500:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5559e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7ad expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1769074847.361673:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x206:0x0]: returning size 4096 00010000:00010000:2.0:1769074847.361775:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5559e lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7ad expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.361788:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074847.361791:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074847.361796:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5559e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7ad expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.361833:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074847.361899:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66728:x1855007617020544:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 444us (858us total) trans 8589944106 rc 301/301 00000100:00100000:2.0:1769074847.361907:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13552 00000100:00100000:1.0:1769074847.367743:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.367835:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617020672 00000100:00100000:0.0:1769074847.367849:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617020672 00000100:00100000:0.0:1769074847.367905:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13553 00000100:00100000:0.0:1769074847.367914:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66728:x1855007617020672:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074847.368002:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368019:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368030:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368041:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368047:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368054:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368275:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368282:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368286:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.368319:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:0.0:1769074847.368322:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074847.368332:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368341:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368345:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368349:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368353:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368357:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.368359:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368364:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368372:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.368374:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.368378:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368383:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555ac lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368401:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368406:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368410:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368415:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.368418:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368422:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368429:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.368431:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.368434:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368439:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555b3 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368449:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368455:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368460:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368464:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.368467:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368472:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368479:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.368481:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.368484:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.368489:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555ba lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074847.368566:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66728:x1855007617020672:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 652us (834us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.368572:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13553 00000100:00100000:3.0:1769074847.371103:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.371430:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617020928 00000100:00100000:1.0:1769074847.371442:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617020928 00000100:00100000:1.0:1769074847.371499:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4739 00000100:00100000:1.0:1769074847.371509:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66728:x1855007617020928:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:1.0:1769074847.371734:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66728:x1855007617020928:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 227us (641us total) trans 8589944107 rc 0/0 00000100:00100000:1.0:1769074847.371742:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4739 00000100:00100000:1.0:1769074847.413920:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.414122:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617021184 00000100:00100000:0.0:1769074847.414133:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617021184 00000100:00100000:0.0:1769074847.414197:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13554 00000100:00100000:0.0:1769074847.414206:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66732:x1855007617021184:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.414241:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.414265:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555c1 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7b4 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.414340:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.414353:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.414963:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.414980:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.414988:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.414993:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415083:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415090:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415096:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415103:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415109:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.415112:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415122:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415135:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.415138:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.415142:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415148:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555c8 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415160:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415166:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415171:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415176:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.415179:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415185:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415191:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80700) 00010000:00010000:0.0:1769074847.415193:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.415195:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415201:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555cf lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.415249:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555c1 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7b4 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.415259:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.415261:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074847.415265:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555c1 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7b4 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.415277:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135600, rc 0) 00000100:00100000:0.0:1769074847.415340:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66732:x1855007617021184:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1134us (1428us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074847.415348:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13554 00000100:00100000:3.0:1769074847.423286:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.423582:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617021440 00000100:00100000:2.0:1769074847.423594:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617021440 00000100:00100000:2.0:1769074847.423676:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13555 00000100:00100000:2.0:1769074847.423687:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66732:x1855007617021440:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.423708:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.423730:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd555d6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7bb expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.424266:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd555dd lrc: 3/1,0 mode: --/CR res: [0x200000405:0x206:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.424282:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd555dd lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x206:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.424292:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd555dd lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x206:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074847.424304:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x206:0x0]: returning size 4096 00010000:00010000:2.0:1769074847.424498:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd555d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7bb expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.424518:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89c00/0x7fe2323adfd555dd lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x206:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7bb expref: 23 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.424527:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x206:0x0].0x0 (ffff89b643a80800) 00010000:00010000:2.0:1769074847.424530:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1769074847.424533:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89c00, rc 0) 00000100:00100000:2.0:1769074847.424596:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66732:x1855007617021440:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 912us (1320us total) trans 8589944108 rc 0/0 00000100:00100000:2.0:1769074847.424605:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13555 00000100:00100000:2.0:1769074847.427958:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.428309:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617021568 00000100:00100000:0.0:1769074847.428319:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617021568 00000100:00100000:0.0:1769074847.428413:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13556 00000100:00100000:0.0:1769074847.428423:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66732:x1855007617021568:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074847.428514:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428531:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428542:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428553:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428560:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428567:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428672:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428681:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428688:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.428723:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:0.0:1769074847.428727:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074847.428739:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428751:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428758:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428764:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428770:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428776:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.428780:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428787:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428799:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.428802:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.428843:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428849:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd555eb lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428866:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428880:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428886:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428892:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.428894:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428900:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428907:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80700) 00010000:00010000:0.0:1769074847.428909:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.428912:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428918:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd555f2 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428927:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428934:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428940:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428944:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.428947:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428952:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428957:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.428959:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.428962:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.428967:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd555f9 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074847.429039:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66732:x1855007617021568:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 617us (1088us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.429046:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13556 00000100:00100000:1.0:1769074847.472960:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.473080:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617021824 00000100:00100000:0.0:1769074847.473091:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617021824 00000100:00100000:0.0:1769074847.473159:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13557 00000100:00100000:0.0:1769074847.473170:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66740:x1855007617021824:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.473191:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.473213:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55600 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7c2 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.473289:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473315:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473327:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473339:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473345:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473351:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473429:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473436:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473442:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473448:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473453:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.473458:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473469:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473481:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.473485:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.473489:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473495:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55607 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473509:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473515:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473520:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473525:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.473527:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473533:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473539:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.473541:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.473544:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473548:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd5560e lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.473579:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55600 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7c2 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.473588:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.473591:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074847.473594:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55600 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7c2 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.473605:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135400, rc 0) 00000100:00100000:0.0:1769074847.473716:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66740:x1855007617021824:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 548us (764us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074847.473723:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13557 00000100:00100000:3.0:1769074847.483546:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.483867:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617021952 00000100:00100000:2.0:1769074847.483880:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617021952 00000100:00100000:2.0:1769074847.483951:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13558 00000100:00100000:2.0:1769074847.483961:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66740:x1855007617021952:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074847.484043:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484073:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484086:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484099:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484107:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484114:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484200:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484210:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484216:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074847.484258:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:2.0:1769074847.484262:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074847.484277:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484290:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484297:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484303:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484309:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484315:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.484319:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484325:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484336:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074847.484338:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.484342:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484348:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55615 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484364:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484369:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484374:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484378:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.484380:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484385:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484391:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effac00) 00010000:00010000:2.0:1769074847.484394:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.484396:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484401:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd5561c lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484421:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484427:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484431:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484436:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.484438:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484443:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484449:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.484451:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.484453:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.484458:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55623 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074847.484535:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66740:x1855007617021952:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 575us (999us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.484544:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13558 00000100:00100000:1.0:1769074847.513852:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.515145:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617022208 00000100:00100000:0.0:1769074847.515159:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617022208 00000100:00100000:0.0:1769074847.515238:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13559 00000100:00100000:0.0:1769074847.515248:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66747:x1855007617022208:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.515269:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.515294:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5562a lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7c9 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.515370:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515383:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515395:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515422:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515429:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515435:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515510:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515518:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515524:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515530:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515536:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.515541:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515548:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515560:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.515563:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.515567:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515573:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55631 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515588:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515594:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515599:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515604:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.515606:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515611:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515618:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.515621:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.515623:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515628:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55638 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.515661:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5562a lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7c9 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.515670:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.515672:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074847.515676:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5562a lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7c9 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.515691:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135400, rc 0) 00000100:00100000:0.0:1769074847.515770:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66747:x1855007617022208:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 522us (1928us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074847.515778:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13559 00000100:00100000:1.0:1769074847.528304:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.528491:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617022336 00000100:00100000:2.0:1769074847.528503:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617022336 00000100:00100000:2.0:1769074847.528561:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13560 00000100:00100000:2.0:1769074847.528572:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66747:x1855007617022336:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074847.528665:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528683:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528694:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528707:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528714:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528720:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528790:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528799:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528839:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074847.528876:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:2.0:1769074847.528880:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074847.528893:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528905:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528916:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528923:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528929:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528935:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.528939:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528947:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528958:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074847.528961:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.528965:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528971:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5563f lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528987:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528993:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.528998:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529004:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.529007:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529013:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529019:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa600) 00010000:00010000:2.0:1769074847.529021:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.529024:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529029:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55646 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529038:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529043:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529048:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529063:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074847.529066:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529072:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529079:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.529081:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074847.529084:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074847.529094:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5564d lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074847.529165:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66747:x1855007617022336:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 594us (870us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.529173:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13560 00000100:00100000:3.0:1769074847.572701:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.572850:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617022592 00000100:00100000:0.0:1769074847.572862:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617022592 00000100:00100000:0.0:1769074847.572951:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13561 00000100:00100000:0.0:1769074847.572960:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66755:x1855007617022592:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.572979:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.573001:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55654 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7d0 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.573088:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573102:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573113:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573124:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573131:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573137:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573215:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573222:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573228:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573234:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573240:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.573245:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573252:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573264:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.573266:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.573271:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573277:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5565b lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573291:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573297:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573302:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573308:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.573311:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573316:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573323:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.573325:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.573327:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573333:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55662 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.573364:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55654 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7d0 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.573374:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.573376:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074847.573379:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55654 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7d0 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.573389:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135400, rc 0) 00000100:00100000:0.0:1769074847.574101:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66755:x1855007617022592:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1138us (1406us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074847.574111:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13561 00000100:00100000:3.0:1769074847.588478:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.590493:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617022720 00000100:00100000:0.0:1769074847.590508:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617022720 00000100:00100000:0.0:1769074847.590646:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13562 00000100:00100000:0.0:1769074847.590656:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66755:x1855007617022720:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074847.590762:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590778:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590789:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590800:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590824:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590831:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590903:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590912:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590918:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.590956:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:0.0:1769074847.590962:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074847.590974:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590988:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.590994:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591001:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591008:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591014:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.591018:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591025:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591037:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80700) 00010000:00010000:0.0:1769074847.591040:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.591044:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591050:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55669 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591077:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591083:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591089:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591094:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.591097:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591102:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591108:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.591110:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.591113:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591117:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55670 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591126:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591131:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591136:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591141:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.591144:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591150:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591156:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.591159:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.591162:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.591171:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55677 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:0.0:1769074847.591256:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66755:x1855007617022720:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 601us (2789us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.591265:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13562 00000100:00100000:3.0:1769074847.618367:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.619108:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617022848 00000100:00100000:0.0:1769074847.619119:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617022848 00000100:00100000:0.0:1769074847.619242:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13563 00000100:00100000:0.0:1769074847.619253:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66757:x1855007617022848:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.619282:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.619309:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5567e lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7d7 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.619378:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619392:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619413:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619426:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619433:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619452:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619525:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619533:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619539:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.619576:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:0.0:1769074847.619580:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074847.619599:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619611:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619620:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619626:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619632:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619639:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619645:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.619649:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619656:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619667:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80f00) 00010000:00010000:0.0:1769074847.619671:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.619676:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619682:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135600/0x7fe2323adfd55685 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619696:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619701:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619706:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619710:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.619713:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619718:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619726:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80700) 00010000:00010000:0.0:1769074847.619728:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074847.619731:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619736:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd5568c lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.619762:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd5567e lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7d7 expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.619776:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7d7 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.619783:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.619785:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074847.619788:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134c00, rc 0) 00000100:00100000:0.0:1769074847.620009:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:66757:x1855007617022848:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 757us (1650us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.620017:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13563 00000100:00100000:3.0:1769074847.623865:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.623910:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617022976 00000100:00100000:3.0:1769074847.623920:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617022976 00000100:00100000:3.0:1769074847.623979:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14494 00000100:00100000:3.0:1769074847.623988:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617022976:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.624007:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.624028:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7de expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074847.624062:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c7:0x0] initial LVB size: 1048576, mtime: 0x6971f09e, atime: 0x0, ctime: 0x6971f09e, blocks: 0x800 00010000:00010000:3.0:1769074847.624083:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.624086:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.624096:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074847.624114:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074847.624127:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb600, rc 0) 00000100:00100000:3.0:1769074847.624191:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617022976:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 203us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.624198:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14494 00000100:00100000:3.0:1769074847.627084:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.627121:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617023104 00000100:00100000:3.0:1769074847.627130:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617023104 00000100:00100000:3.0:1769074847.627185:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14495 00000100:00100000:3.0:1769074847.627195:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617023104:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.627369:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.627395:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7e5 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074847.627584:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c8:0x0] initial LVB size: 2097152, mtime: 0x6971f09e, atime: 0x0, ctime: 0x6971f09e, blocks: 0x800 00010000:00010000:3.0:1769074847.627605:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.627607:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.627618:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074847.627636:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 10 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074847.627654:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbe00, rc 0) 00000100:00100000:3.0:1769074847.627711:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2407:x1855007617023104:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 518us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.627718:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14495 00000100:00100000:3.0:1769074847.632744:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.632985:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617023360 00000100:00100000:0.0:1769074847.632996:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617023360 00000100:00100000:0.0:1769074847.633065:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13564 00000100:00100000:0.0:1769074847.633074:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:66757:x1855007617023360:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.633094:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.633116:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd556a8 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7ec expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.633363:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd556af lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.633380:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd556af lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.633390:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd556af lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:0.0:1769074847.633690:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00010000:00010000:0.0:1769074847.633833:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd556a8 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7ec expref: 27 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.633852:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd556af lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7ec expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.633861:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.633863:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074847.633866:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135400, rc 0) 00000100:00100000:0.0:1769074847.634288:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:66757:x1855007617023360:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1213us (1553us total) trans 8589944109 rc 0/0 00000100:00100000:0.0:1769074847.634298:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13564 00000100:00100000:2.0:1769074847.641081:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.641141:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.641223:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617023488 00000100:00100000:0.0:1769074847.641234:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617023488 00000100:00100000:0.0:1769074847.641323:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1045 00000100:00100000:0.0:1769074847.641332:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2406:x1855007617023488:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074847.641351:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074847.641375:0:6498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd556af lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7ec expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.641393:0:6498:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.641409:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074847.641470:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2406:x1855007617023488:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.641478:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1045 00000100:00100000:3.0:1769074847.643365:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617023744 00000100:00100000:3.0:1769074847.643378:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617023744 00000100:00100000:3.0:1769074847.643641:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13565 00000100:00100000:3.0:1769074847.643652:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:66757:x1855007617023744:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.643677:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.643697:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556bd lrc: 2/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7f3 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.643838:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd556c4 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074847.643853:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd556c4 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074847.643864:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd556c4 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074847.643886:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00010000:00010000:3.0:1769074847.643999:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556bd lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7f3 expref: 28 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.644017:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd556c4 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7f3 expref: 27 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.644030:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074847.644032:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769074847.644036:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67feba000, rc 0) 00000100:00100000:3.0:1769074847.644119:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:66757:x1855007617023744:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 469us (2977us total) trans 8589944110 rc 0/0 00000100:00100000:3.0:1769074847.644127:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13565 00000100:00100000:3.0:1769074847.648257:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.648303:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.648389:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617023872 00000100:00100000:1.0:1769074847.648411:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617023872 00000100:00100000:1.0:1769074847.648465:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024000 00000100:00100000:1.0:1769074847.648469:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024000 00000100:00100000:1.0:1769074847.648494:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14496 00000100:00100000:1.0:1769074847.648504:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2407:x1855007617023872:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074847.648548:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.648566:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074847.648575:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.648577:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.648580:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.648615:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074847.648629:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 11 pid: 28955 timeout: 1935 lvb_type: 1 00000100:00100000:1.0:1769074847.648642:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625233536:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074847.648717:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452780 going to sleep for 11 seconds 00000100:00100000:1.0:1769074847.648737:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14497 00000100:00100000:1.0:1769074847.648745:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2407:x1855007617024000:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074847.648763:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.648773:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074847.648781:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.648782:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.648785:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.648803:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 11 pid: 27843 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074847.648851:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 11 pid: 27843 timeout: 1935 lvb_type: 1 00000100:00100000:1.0:1769074847.648861:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625233664:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074847.648899:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074847.653019:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625233536:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074847.653068:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074847.653248:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625233664:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074847.653263:0:9048:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074847.653338:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.653380:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.653465:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024128 00000100:00100000:0.0:1769074847.653472:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024128 00010000:00010000:0.0:1769074847.653503:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 12 pid: 28955 timeout: 1935 lvb_type: 1 00000100:00100000:0.0:1769074847.653542:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024256 00000100:00100000:0.0:1769074847.653544:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024256 00010000:00010000:0.0:1769074847.653556:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 13 pid: 27843 timeout: 1935 lvb_type: 1 00000100:00100000:0.0:1769074847.653583:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7215 00000100:00100000:0.0:1769074847.653592:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2408:x1855007617024128:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074847.653607:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074847.653632:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 13 pid: 28955 timeout: 1935 lvb_type: 1 00010000:00010000:0.0:1769074847.653645:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 13 pid: 28955 timeout: 1935 lvb_type: 1 00010000:00010000:0.0:1769074847.653670:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb600/0x7fe2323adfd5569a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7de expref: 13 pid: 28955 timeout: 1935 lvb_type: 1 00010000:00010000:0.0:1769074847.653686:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.653688:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.653693:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.653704:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.653725:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074847.653779:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617024128:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 188us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.653787:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7215 00000100:00100000:0.0:1769074847.653855:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7216 00000100:00100000:0.0:1769074847.653861:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617024256:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074847.653879:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074847.653888:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 11 pid: 27843 timeout: 1935 lvb_type: 1 00010000:00010000:0.0:1769074847.653899:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 11 pid: 27843 timeout: 1935 lvb_type: 1 00010000:00010000:0.0:1769074847.653911:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febbe00/0x7fe2323adfd556a1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7e5 expref: 11 pid: 27843 timeout: 1935 lvb_type: 1 00010000:00010000:0.0:1769074847.653920:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.653921:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.653924:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.653931:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.653946:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074847.653967:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617024256:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 106us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.653973:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7216 00010000:00010000:3.0:1769074847.654189:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654199:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654219:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654227:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654234:0:28955:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654241:0:28955:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074847.654245:0:28955:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654253:0:28955:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654264:0:28955:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.654271:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549242000/0x7fe2323adfd556d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074847.654322:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617023872:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 5819us (6075us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.654330:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14496 00010000:00010000:2.0:1769074847.654603:0:9048:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654613:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654628:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654635:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654643:0:9048:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654654:0:9048:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074847.654657:0:9048:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654663:0:9048:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654674:0:9048:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.654681:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd556d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074847.654736:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617024000:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 5991us (6432us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.654744:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14497 00000100:00100000:1.0:1769074847.660179:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.660316:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024512 00000100:00100000:3.0:1769074847.660326:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024512 00000100:00100000:3.0:1769074847.660385:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13566 00000100:00100000:3.0:1769074847.660406:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:66757:x1855007617024512:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074847.660473:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1769074847.660502:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd55693 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7d7 expref: 27 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.660523:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55566 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c798 expref: 26 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.660539:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074847.660544:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074847.660573:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074847.660588:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074847.660597:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00080000:3.0:1769074847.660775:0:6514:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2ca:0x0], index 1: 0 00000040:00080000:3.0:1769074847.661546:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2465, 88 off210632 00000004:00080000:3.0:1769074847.661556:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1835503748658ns, next at 1809352413082ns, rc = 0 00000040:00080000:3.0:1769074847.661621:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2466, 88 off210720 00000004:00080000:3.0:1769074847.661626:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1835503819455ns, next at 1809352413082ns, rc = 0 00000040:00080000:3.0:1769074847.665833:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2469, 88 off211224 00000004:00080000:3.0:1769074847.665854:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1835508046650ns, next at 1819234810192ns, rc = 0 00000040:00080000:3.0:1769074847.665882:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2470, 88 off211312 00000004:00080000:3.0:1769074847.665886:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1835508079172ns, next at 1819234810192ns, rc = 0 00000001:00080000:3.0:1769074847.669647:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944111 is committed 00000004:00080000:3.0:1769074847.669657:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1966 committed 00000004:00080000:3.0:1769074847.669677:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1965 committed 00000004:00080000:3.0:1769074847.669680:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1964 committed 00000004:00080000:3.0:1769074847.669687:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1963 committed 00000004:00010000:3.0:1769074847.669748:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000cff33ab9 reply state 00000000ff7e3f79 transno 8589944111 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074847.669771:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074847.669780:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074847.669783:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:3.0:1769074847.669787:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00080000:3.0:1769074847.669908:0:6514:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1769074847.669932:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625233920:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074847.669985:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074847.670074:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074847.670081:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625234048:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074847.670104:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074847.670132:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625233792:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074847.670153:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074847.670170:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074847.670174:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66757:x1855007617024512:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 9777us (10002us total) trans 8589944111 rc 0/0 00000100:00100000:2.0:1769074847.670175:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625234176:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074847.670184:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13566 00000100:00100000:2.0:1769074847.670193:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074847.670225:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625233920 00000100:00100000:2.0:1769074847.670232:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625233920 00000100:00100000:2.0:1769074847.670271:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625234048 00000100:00100000:2.0:1769074847.670275:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625234048 00000100:00100000:2.0:1769074847.670296:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625233792 00000100:00100000:2.0:1769074847.670299:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625233792 00000100:00100000:2.0:1769074847.670314:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625234176 00000100:00100000:2.0:1769074847.670316:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625234176 00000100:00100000:2.0:1769074847.670333:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14498 00000100:00100000:2.0:1769074847.670340:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625233920:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074847.670504:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625233920:12345-0@lo:2:osp-syn-1-0.0 Request processed in 163us (517us total) trans 17179874625 rc 0/0 00000100:00100000:2.0:1769074847.670510:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14498 00000100:00100000:2.0:1769074847.670524:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14499 00000100:00100000:2.0:1769074847.670529:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625234048:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074847.670623:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625234048:12345-0@lo:2:osp-syn-0-0.0 Request processed in 93us (517us total) trans 21474841299 rc 0/0 00000100:00100000:2.0:1769074847.670629:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14499 00000100:00100000:2.0:1769074847.670641:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14500 00000100:00100000:2.0:1769074847.670646:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625233792:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074847.670720:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+8:3650:x1855007625233792:12345-0@lo:2:osp-syn-1-0.0 Request processed in 73us (565us total) trans 17179874626 rc 0/0 00000100:00100000:2.0:1769074847.670726:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14500 00000100:00100000:2.0:1769074847.670738:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14501 00000100:00100000:2.0:1769074847.670744:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625234176:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074847.670844:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625234176:12345-0@lo:2:osp-syn-0-0.0 Request processed in 100us (649us total) trans 21474841300 rc 0/0 00000100:00100000:2.0:1769074847.670850:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14501 00000100:00100000:2.0:1769074847.670896:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841296 gen 2 00000004:00080000:2.0:1769074847.670901:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef4e00, transno 21474841295 00000004:00080000:2.0:1769074847.670909:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547ef5180, transno 21474841296 00000100:00100000:2.0:1769074847.670915:0:3651:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b547599180 x1855007625234048/t21474841299(21474841299) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074863 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1769074847.670926:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547599180/2, rc 0, transno 21474841299 00000100:00100000:2.0:1769074847.670930:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625234048:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074847.670939:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547598a80/2, rc 0, transno 17179874625 00000100:00100000:2.0:1769074847.670943:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625233920:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1769074847.671023:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547598e00/2, rc 0, transno 21474841300 00000100:00100000:2.0:1769074847.671027:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625234176:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074847.671034:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759ad80/2, rc 0, transno 17179874626 00000100:00100000:2.0:1769074847.671037:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625233792:0@lo:2:osp-syn-1-0.0 00010000:00010000:0.0:1769074847.672647:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.672662:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.672669:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.672674:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074847.672678:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.672685:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.672699:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074847.672702:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074847.672706:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074847.672712:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb200/0x7fe2323adfd556e0 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074847.674837:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.674909:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024640 00000100:00100000:2.0:1769074847.674919:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024640 00000100:00100000:2.0:1769074847.674981:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13567 00000100:00100000:2.0:1769074847.674991:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66757:x1855007617024640:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.675014:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.675034:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c7fa expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.675099:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.675117:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 20 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.675133:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.675137:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074847.675140:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88000, rc 0) 00000100:00100000:2.0:1769074847.675196:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66757:x1855007617024640:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 207us (388us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.675203:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13567 00000100:00100000:3.0:1769074847.679088:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.679309:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024768 00000100:00100000:2.0:1769074847.679320:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024768 00000100:00100000:2.0:1769074847.679435:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13568 00000100:00100000:2.0:1769074847.679456:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66757:x1855007617024768:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074847.679477:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074847.679499:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd556ee lrc: 2/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c801 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.679598:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd556f5 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.679614:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd556f5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.679626:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd556f5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074847.679654:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097153 00000004:00200000:2.0:1769074847.679658:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074847.679675:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd556f5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.679702:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd556ee lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c801 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.679719:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd556f5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c801 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.679727:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.679730:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074847.679733:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88c00, rc 0) 00000100:00100000:2.0:1769074847.679801:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66757:x1855007617024768:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 347us (886us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.679837:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13568 00000100:00100000:1.0:1769074847.684445:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.684485:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.684499:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617024896 00000100:00100000:3.0:1769074847.684510:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617024896 00000100:00100000:3.0:1769074847.684593:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025024 00000100:00100000:3.0:1769074847.684597:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025024 00000100:00100000:0.0:1769074847.684652:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14502 00000100:00100000:0.0:1769074847.684665:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617024896:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.684684:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.684706:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c808 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074847.684730:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c7:0x0] initial LVB size: 1048576, mtime: 0x6971f09e, atime: 0x0, ctime: 0x6971f09e, blocks: 0x800 00010000:00010000:0.0:1769074847.684750:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.684753:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.684764:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 10 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074847.684781:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 10 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074847.684794:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135400, rc 0) 00000100:00100000:3.0:1769074847.685153:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14503 00000100:00100000:3.0:1769074847.685162:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617025024:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074847.685169:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074847.685180:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c80f expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074847.685193:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c8:0x0] initial LVB size: 2097152, mtime: 0x6971f09e, atime: 0x0, ctime: 0x6971f09e, blocks: 0x800 00010000:00010000:3.0:1769074847.685202:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.685204:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.685209:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074847.686085:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074847.686102:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:1.0:1769074847.686105:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617024896:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1443us (1666us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074847.686113:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14502 00000100:00100000:3.0:1769074847.686126:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617025024:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 967us (1641us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.686131:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14503 00000100:00100000:3.0:1769074847.761509:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.761557:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.768003:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025152 00000100:00100000:0.0:1769074847.768021:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025152 00000100:00100000:0.0:1769074847.768098:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025280 00000100:00100000:0.0:1769074847.768102:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025280 00000100:00100000:0.0:1769074847.768126:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14504 00000100:00100000:0.0:1769074847.768135:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617025152:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:3.0:1769074847.768453:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14505 00000100:00100000:3.0:1769074847.768464:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617025280:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:3.0:1769074847.768537:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617025280:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 74us (6979us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074847.768544:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14505 00000100:00100000:0.0:1769074847.768613:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617025152:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 478us (7116us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.768621:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14504 00000100:00100000:0.0:1769074847.778706:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.778754:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.779555:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025536 00000100:00100000:2.0:1769074847.779568:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025536 00000100:00100000:2.0:1769074847.779627:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025408 00000100:00100000:2.0:1769074847.779631:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025408 00000100:00100000:2.0:1769074847.779664:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14506 00000100:00100000:2.0:1769074847.779674:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617025536:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:2.0:1769074847.779796:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617025536:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 123us (1099us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.779832:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14506 00000100:00100000:2.0:1769074847.779850:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14507 00000100:00100000:2.0:1769074847.779856:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617025408:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:2.0:1769074847.779894:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617025408:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 38us (1139us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.779904:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14507 00000100:00100000:1.0:1769074847.788405:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.788445:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.788592:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025792 00000100:00100000:3.0:1769074847.788603:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025792 00000100:00100000:3.0:1769074847.788682:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025664 00000100:00100000:3.0:1769074847.788686:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025664 00000100:00100000:3.0:1769074847.788719:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8751 00000100:00100000:3.0:1769074847.788736:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617025792:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074847.788773:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.788790:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.788792:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.788797:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.788845:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074847.791333:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8752 00000100:00100000:0.0:1769074847.791360:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617025664:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:0.0:1769074847.791422:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.791441:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.791443:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.791449:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.791469:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818752:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818771:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818780:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818787:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074847.818791:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818801:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818860:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.818867:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd5570a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074847.818947:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617025792:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 30213us (30556us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074847.818957:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8751 00010000:00010000:0.0:1769074847.846359:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.846379:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.846389:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.846549:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074847.846554:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.846564:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.846584:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.846591:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55711 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074847.846691:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617025664:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 55338us (58238us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1769074847.846705:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8752 00000100:00100000:0.0:1769074847.872447:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.873244:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617025920 00000100:00100000:3.0:1769074847.873257:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617025920 00000100:00100000:3.0:1769074847.873339:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8753 00000100:00100000:3.0:1769074847.873349:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617025920:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:3.0:1769074847.873969:0:16437:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1769074847.873982:0:16437:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.874067:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.874069:0:16437:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074847.874075:0:16437:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074847.874091:0:16437:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000001:00080000:3.0:1769074847.915230:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874627 is committed 00000001:00080000:3.0:1769074847.915249:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874626 is committed 00000001:00080000:3.0:1769074847.915258:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874623 is committed 00002000:00010000:2.0:1769074847.915496:0:16437:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:2.0:1769074847.915501:0:16437:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb mtime from disk: 0 -> 1769074847 00002000:00010000:2.0:1769074847.915503:0:16437:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb ctime from disk: 0 -> 1769074847 00002000:00010000:2.0:1769074847.915505:0:16437:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb blocks from disk: 0 -> 4096 00010000:00010000:2.0:1769074847.915520:0:16437:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.915531:0:16437:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.915538:0:16437:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.915545:0:16437:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074847.915550:0:16437:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.915558:0:16437:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.915578:0:16437:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.915585:0:16437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b543a29000/0x7fe2323adfd55718 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16437 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074847.915692:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617025920:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 42338us (43248us total) trans 17179874627 rc 0/0 00000100:00100000:2.0:1769074847.915705:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8753 00000100:00100000:1.0:1769074847.954132:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.954248:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617026176 00000100:00100000:2.0:1769074847.954259:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617026176 00000100:00100000:2.0:1769074847.954326:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13569 00000100:00100000:2.0:1769074847.954336:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:66757:x1855007617026176:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074847.954400:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074847.954429:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd556f5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c801 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.954455:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.954461:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074847.954648:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,1 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.954662:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.954671:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000001:00080000:2.0:1769074847.954712:0:6514:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944112 00000004:00010000:2.0:1769074847.954733:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000adbe6484 reply state 0000000072691448 transno 8589944113 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.954743:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074847.954748:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074847.954751:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074847.954754:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074847.954913:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66757:x1855007617026176:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 578us (789us total) trans 8589944113 rc 0/0 00000100:00100000:2.0:1769074847.954920:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13569 00000020:00080000:2.0:1769074847.954925:0:6514:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1769074847.959126:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074847.959242:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617026304 00000100:00100000:1.0:1769074847.959253:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617026304 00000100:00100000:1.0:1769074847.959317:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14508 00000100:00100000:1.0:1769074847.959326:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:66757:x1855007617026304:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074847.959344:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074847.959365:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c816 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074847.959388:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2c9:0x0] initial LVB size: 2097152, mtime: 0x6971f09f, atime: 0x0, ctime: 0x6971f09f, blocks: 0x1000 00010000:00010000:1.0:1769074847.959420:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.959423:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.959432:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.959446:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.959458:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242a00, rc 0) 00000100:00100000:1.0:1769074847.959515:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66757:x1855007617026304:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 189us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074847.959522:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14508 00000100:00100000:3.0:1769074847.964363:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.965670:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617026432 00000100:00100000:0.0:1769074847.965684:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617026432 00000100:00100000:0.0:1769074847.965795:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14509 00000100:00100000:0.0:1769074847.965830:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:66757:x1855007617026432:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.965849:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.965882:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c81d expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1769074847.965907:0:27843:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2ca:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1769074847.965937:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.965940:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.965949:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.965966:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.965979:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135e00, rc 0) 00000100:00100000:0.0:1769074847.966054:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:66757:x1855007617026432:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 241us (1691us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.966062:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14509 00000100:00100000:2.0:1769074847.970654:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.970705:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.970794:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617026560 00000100:00100000:3.0:1769074847.970803:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617026560 00002000:00010000:3.0:1769074847.971014:0:16437:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/713 (2097152->18446744073709551615) 00002000:00100000:3.0:1769074847.971020:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dca80 with x1855007617026560 ext(2097152->18446744073709551615) 00002000:00010000:3.0:1769074847.971032:0:16437:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dca80. 00000100:00100000:3.0:1769074847.971063:0:16437:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617026688 00000100:00100000:3.0:1769074847.971067:0:16437:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617026688 00002000:00010000:3.0:1769074847.971077:0:16437:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/714 (0->18446744073709551615) 00002000:00100000:3.0:1769074847.971080:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dea00 with x1855007617026688 ext(0->18446744073709551615) 00002000:00010000:3.0:1769074847.971085:0:16437:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dea00. 00000100:00100000:3.0:1769074847.971111:0:16437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8754 00000100:00100000:3.0:1769074847.971120:0:16437:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617026560:12345-192.168.204.60@tcp:10:lfs.0 00000100:00100000:2.0:1769074847.971329:0:16436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8755 00000100:00100000:2.0:1769074847.971339:0:16436:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617026688:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:2.0:1769074847.971485:0:16436:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1769074847.971489:0:16436:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb mtime from disk: 0 -> 1769074847 00002000:00010000:2.0:1769074847.971491:0:16436:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074847.971493:0:16436:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb ctime from disk: 0 -> 1769074847 00002000:00010000:2.0:1769074847.971496:0:16436:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074847.971559:0:16436:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617026688:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 222us (852us total) trans 17179874628 rc 0/0 00000100:00100000:2.0:1769074847.971567:0:16436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8755 00002000:00010000:2.0:1769074847.971570:0:16436:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/714 (0->18446744073709551615) 00002000:00100000:2.0:1769074847.971573:0:16436:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dea00 with x1855007617026688 ext(0->18446744073709551615) 00002000:00010000:2.0:1769074847.971577:0:16436:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dea00. 00002000:00010000:3.0:1769074847.972594:0:16437:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:3.0:1769074847.972602:0:16437:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb mtime from disk: 1769074847 -> 1769074847 00002000:00010000:3.0:1769074847.972604:0:16437:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769074847.972606:0:16437:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb ctime from disk: 1769074847 -> 1769074847 00002000:00010000:3.0:1769074847.972608:0:16437:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:3.0:1769074847.972659:0:16437:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617026560:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 1536us (2012us total) trans 17179874629 rc 0/0 00000100:00100000:3.0:1769074847.972667:0:16437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8754 00002000:00010000:3.0:1769074847.972671:0:16437:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/713 (2097152->18446744073709551615) 00002000:00100000:3.0:1769074847.972674:0:16437:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54d1dca80 with x1855007617026560 ext(2097152->18446744073709551615) 00002000:00010000:3.0:1769074847.972680:0:16437:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54d1dca80. 00000100:00100000:2.0:1769074847.976156:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074847.976341:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617026944 00000100:00100000:3.0:1769074847.976352:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617026944 00000100:00100000:3.0:1769074847.976420:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13570 00000100:00100000:3.0:1769074847.976431:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:66757:x1855007617026944:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074847.976552:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.976569:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.976582:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00010000:3.0:1769074847.976690:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008e43a5ae reply state 0000000061a8976b transno 8589944114 ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.976703:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074847.976710:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074847.976712:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074847.976715:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074847.977097:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:66757:x1855007617026944:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 667us (951us total) trans 8589944114 rc 0/0 00000100:00100000:3.0:1769074847.977105:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13570 00000100:00100000:2.0:1769074847.980938:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074847.981085:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027072 00000100:00100000:0.0:1769074847.981095:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027072 00000100:00100000:0.0:1769074847.981180:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14510 00000100:00100000:0.0:1769074847.981190:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66757:x1855007617027072:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074847.981218:0:27843:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074847.981243:0:27843:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c824 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.981275:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 11 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074847.981283:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.981285:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074847.981297:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.981329:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 11 pid: 36516 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1769074847.981342:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 11 pid: 36516 timeout: 1935 lvb_type: 1 00000100:00100000:0.0:1769074847.981356:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625234304:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074847.981436:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1769074847.985042:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.985157:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027200 00000100:00100000:2.0:1769074847.985167:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027200 00010000:00010000:2.0:1769074847.985211:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 12 pid: 36516 timeout: 1935 lvb_type: 1 00000100:00100000:2.0:1769074847.985257:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7217 00000100:00100000:2.0:1769074847.985267:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617027200:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074847.985283:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074847.985306:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 12 pid: 36516 timeout: 1935 lvb_type: 1 00010000:00010000:2.0:1769074847.985318:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 12 pid: 36516 timeout: 1935 lvb_type: 1 00010000:00010000:2.0:1769074847.985342:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074847.985345:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074847.985351:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.985363:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074847.985411:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 12 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074847.985425:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625234432:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074847.985509:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b681626240 going to sleep for 11 seconds 00010000:00010000:0.0:1769074847.986891:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549135400/0x7fe2323adfd556fc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c808 expref: 12 pid: 36516 timeout: 1935 lvb_type: 1 00000100:00100000:0.0:1769074847.986919:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625234304:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074847.986946:0:27843:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 11 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074847.986964:0:27843:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134200, rc 0) 00000100:00100000:0.0:1769074847.987027:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66757:x1855007617027072:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 5838us (6099us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074847.987035:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14510 00000100:00100000:3.0:1769074847.992603:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074847.992695:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625234432:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074847.992723:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074847.992780:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617027200:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 7514us (7748us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074847.992792:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7217 00000100:00100000:1.0:1769074847.995146:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027328 00000100:00100000:1.0:1769074847.995162:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027328 00000100:00100000:1.0:1769074847.995237:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14511 00000100:00100000:1.0:1769074847.995247:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:66757:x1855007617027328:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074847.995265:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074847.995284:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c82b expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.995317:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074847.995325:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.995328:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074847.995331:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [1048576->68157439] (req 1048576->68157439) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074847.995366:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 11 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074847.995379:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 11 pid: 28955 timeout: 1935 lvb_type: 1 00000100:00100000:1.0:1769074847.995410:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625234560:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074847.995501:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452f00 going to sleep for 11 seconds 00000100:00100000:0.0:1769074848.000405:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.001386:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027456 00000100:00100000:2.0:1769074848.001408:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027456 00010000:00010000:2.0:1769074848.001464:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 12 pid: 28955 timeout: 1935 lvb_type: 1 00000100:00100000:2.0:1769074848.001552:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7218 00000100:00100000:2.0:1769074848.001562:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617027456:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074848.001577:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074848.001600:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 12 pid: 28955 timeout: 1935 lvb_type: 1 00010000:00010000:2.0:1769074848.001615:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 12 pid: 28955 timeout: 1935 lvb_type: 1 00010000:00010000:2.0:1769074848.001642:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074848.001644:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074848.001651:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.001661:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.001700:0:36300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 12 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074848.001715:0:36300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625234688:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074848.001802:0:36300:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b681626240 going to sleep for 11 seconds 00010000:00010000:1.0:1769074848.004192:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55703 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c80f expref: 12 pid: 28955 timeout: 1935 lvb_type: 1 00000100:00100000:1.0:1769074848.004385:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625234560:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074848.004427:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074848.004441:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243200, rc 0) 00000100:00100000:1.0:1769074848.004525:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:66757:x1855007617027328:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9279us (11932us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074848.004534:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14511 00000100:00100000:0.0:1769074848.009497:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074848.009526:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074848.009597:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027584 00000100:00100000:1.0:1769074848.009607:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027584 00000100:00100000:1.0:1769074848.009654:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027712 00000100:00100000:1.0:1769074848.009657:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027712 00000100:00100000:1.0:1769074848.009675:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14512 00000100:00100000:1.0:1769074848.009683:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617027584:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:1.0:1769074848.009841:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:1.0:1769074848.009845:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb mtime from disk: 1769074846 -> 1769074846 00002000:00010000:1.0:1769074848.009848:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb atime from disk: 0 -> 1769074846 00002000:00010000:1.0:1769074848.009851:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb ctime from disk: 1769074846 -> 1769074847 00002000:00010000:1.0:1769074848.009853:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074848.009916:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2408:x1855007617027584:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 232us (428us total) trans 21474841301 rc 0/0 00000100:00100000:1.0:1769074848.009923:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14512 00000100:00100000:1.0:1769074848.009943:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14513 00000100:00100000:1.0:1769074848.009948:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617027712:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:1.0:1769074848.010004:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074848.010007:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb mtime from disk: 1769074846 -> 1769074846 00002000:00010000:1.0:1769074848.010010:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb atime from disk: 0 -> 1769074846 00002000:00010000:1.0:1769074848.010012:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb ctime from disk: 1769074846 -> 1769074847 00002000:00010000:1.0:1769074848.010014:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074848.010032:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2408:x1855007617027712:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 84us (506us total) trans 21474841302 rc 0/0 00000100:00100000:1.0:1769074848.010038:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14513 00000100:00100000:2.0:1769074848.010172:0:36300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:36300:1855007625234688:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074848.010195:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074848.010226:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617027456:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8666us (9837us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.010234:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7218 00000100:00100000:2.0:1769074848.016357:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074848.016588:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617027968 00000100:00100000:1.0:1769074848.016599:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617027968 00000100:00100000:1.0:1769074848.016697:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4740 00000100:00100000:1.0:1769074848.016708:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c0c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:66757:x1855007617027968:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:1.0:1769074848.016771:0:6515:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x200000405:0x207:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd556c4 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7f3 expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074848.016850:0:6515:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074848.016875:0:6515:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074848.016883:0:6515:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 4/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:1.0:1769074848.016918:0:6515:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074848.016930:0:6515:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 25 pid: 6514 timeout: 1935 lvb_type: 0 00000100:00100000:1.0:1769074848.016944:0:6515:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625234816:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074848.017027:0:6515:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452f00 going to sleep for 11 seconds 00000100:00100000:3.0:1769074848.020306:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074848.020434:0:6515:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:6515:1855007625234816:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074848.020471:0:6515:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000100:00100000:2.0:1769074848.020780:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617028096 00000100:00100000:2.0:1769074848.020788:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617028096 00010000:00010000:2.0:1769074848.020853:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 26 pid: 6514 timeout: 1935 lvb_type: 0 00000100:00100000:2.0:1769074848.020895:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7219 00000100:00100000:2.0:1769074848.020905:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007617028096:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074848.020922:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074848.020928:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 26 pid: 6514 timeout: 1935 lvb_type: 0 00010000:00010000:2.0:1769074848.020940:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 26 pid: 6514 timeout: 1935 lvb_type: 0 00010000:00010000:2.0:1769074848.020963:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd556e7 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7fa expref: 26 pid: 6514 timeout: 1935 lvb_type: 0 00010000:00010000:2.0:1769074848.020978:0:36300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.020983:0:36300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.020993:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.021001:0:36300:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.021024:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074848.021085:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007617028096:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 181us (787us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.021093:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7219 00010000:00010000:2.0:1769074848.021117:0:6515:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.021123:0:6515:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00000040:00080000:2.0:1769074848.021360:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2467, 88 off210808 00000004:00080000:2.0:1769074848.021370:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1835863562475ns, next at 1809352413082ns, rc = 0 00000040:00080000:2.0:1769074848.021732:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2468, 88 off210896 00000004:00080000:2.0:1769074848.021741:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1835863934203ns, next at 1809352413082ns, rc = 0 00000040:00080000:2.0:1769074848.021774:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2471, 88 off211400 00000004:00080000:2.0:1769074848.021781:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1835863973795ns, next at 1819234810192ns, rc = 0 00000040:00080000:2.0:1769074848.021840:0:6515:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2472, 88 off211488 00000004:00080000:2.0:1769074848.021845:0:6515:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1835864037955ns, next at 1819234810192ns, rc = 0 00000001:00080000:3.0:1769074848.025684:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944115 is committed 00000004:00080000:3.0:1769074848.025696:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1968 committed 00000004:00080000:3.0:1769074848.025747:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1967 committed 00010000:00010000:0.0:1769074848.025785:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025799:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025829:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025836:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074848.025841:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025847:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025871:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074848.025874:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074848.025879:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025884:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5571f lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.025907:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074848.025913:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074848.025918:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074848.025923:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074848.025926:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074848.025932:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074848.025940:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074848.025942:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074848.025945:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074848.025950:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28e00/0x7fe2323adfd55734 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:1.0:1769074848.030073:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625235072:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074848.030147:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.030226:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235072 00000100:00100000:1.0:1769074848.030233:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235072 00000100:00080000:1.0:1769074848.030261:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074848.030303:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14514 00000100:00100000:1.0:1769074848.030312:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625235072:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074848.030500:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625234944:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074848.030533:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074848.030564:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625234944 00000100:00100000:0.0:1769074848.030568:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625234944 00000100:00100000:0.0:1769074848.030592:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14515 00000100:00100000:0.0:1769074848.030599:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625234944:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1769074848.030707:0:27843:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1769074848.030711:0:27843:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb mtime from disk: 1769074847 -> 1769074847 00002000:00010000:0.0:1769074848.030713:0:27843:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1769074848.030716:0:27843:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb ctime from disk: 1769074847 -> 1769074847 00002000:00010000:0.0:1769074848.030718:0:27843:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2c9:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:0.0:1769074848.030757:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625234944:12345-0@lo:2:osp-syn-1-0.0 Request processed in 158us (222us total) trans 17179874630 rc 0/0 00000100:00100000:0.0:1769074848.030763:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14515 00000100:00100000:0.0:1769074848.030797:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874626 gen 2 00000004:00080000:0.0:1769074848.030802:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54c844000, transno 17179874624 00000004:00080000:0.0:1769074848.030829:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547598a80, transno 17179874625 00000004:00080000:0.0:1769074848.030833:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759ad80, transno 17179874626 00000100:00100000:0.0:1769074848.030839:0:3650:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b67a271c00 x1855007625234944/t17179874630(17179874630) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074864 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:0.0:1769074848.030849:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a271c00/2, rc 0, transno 17179874630 00000100:00100000:0.0:1769074848.030854:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625234944:0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074848.031117:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1769074848.031120:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb mtime from disk: 1769074847 -> 1769074847 00002000:00010000:1.0:1769074848.031122:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074848.031124:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb ctime from disk: 1769074847 -> 1769074847 00002000:00010000:1.0:1769074848.031126:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2ca:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1769074848.031214:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625235072:12345-0@lo:2:osp-syn-1-0.0 Request processed in 902us (1065us total) trans 17179874631 rc 0/0 00000100:00100000:1.0:1769074848.031224:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14514 00000004:00080000:1.0:1769074848.031599:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 2 in progress, 1 in flight, 3 done 00000004:00080000:3.0:1769074848.031709:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1966 committed 00000004:00080000:3.0:1769074848.031720:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1965 committed 00000100:00100000:3.0:1769074848.032142:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625235328:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074848.032184:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00010000:2.0:1769074848.032272:0:6515:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000003b61fbd7 reply state 0000000065ccb37b transno 8589944115 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032292:0:6515:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032301:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.032304:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:2.0:1769074848.032308:0:6515:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074848.032314:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.032317:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074848.032324:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba000/0x7fe2323adfd556c4 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c7f3 expref: 19 pid: 6514 timeout: 0 lvb_type: 0 00010000:00080000:2.0:1769074848.032574:0:6515:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1769074848.032589:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c0c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66757:x1855007617027968:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 15882us (16241us total) trans 8589944115 rc 0/0 00000100:00100000:2.0:1769074848.032598:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4740 00010000:00010000:2.0:1769074848.032654:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032661:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032669:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032675:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.032680:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032687:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032697:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.032699:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074848.032703:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00010000:00010000:2.0:1769074848.032708:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55749 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6515 initiator: MDT0 00000100:00100000:1.0:1769074848.033631:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625235200:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074848.033676:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.033728:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235328 00000100:00100000:1.0:1769074848.033736:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235328 00000100:00080000:1.0:1769074848.033759:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074848.033792:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235200 00000100:00100000:1.0:1769074848.033795:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235200 00000100:00100000:1.0:1769074848.033852:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14516 00000100:00100000:1.0:1769074848.033860:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3649:x1855007625235328:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074848.033994:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074848.033998:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb mtime from disk: 1769074846 -> 1769074846 00002000:00010000:1.0:1769074848.034000:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb atime from disk: 1769074846 -> 1769074846 00002000:00010000:1.0:1769074848.034003:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb ctime from disk: 1769074847 -> 1769074847 00002000:00010000:1.0:1769074848.034005:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c8:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074848.034167:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14517 00000100:00100000:2.0:1769074848.034176:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625235200:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1769074848.034250:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:2.0:1769074848.034254:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb mtime from disk: 1769074846 -> 1769074846 00002000:00010000:2.0:1769074848.034256:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb atime from disk: 1769074846 -> 1769074846 00002000:00010000:2.0:1769074848.034258:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb ctime from disk: 1769074847 -> 1769074847 00002000:00010000:2.0:1769074848.034260:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c7:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074848.034291:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+10:3652:x1855007625235200:12345-0@lo:2:osp-syn-0-0.0 Request processed in 116us (614us total) trans 21474841304 rc 0/0 00000100:00100000:2.0:1769074848.034297:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14517 00000004:00080000:0.0:1769074848.036671:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a273800/2, rc 0, transno 17179874631 00000100:00100000:0.0:1769074848.036685:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625235072:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1769074848.037377:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.037523:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617028224 00000100:00100000:3.0:1769074848.037532:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617028224 00000100:00100000:3.0:1769074848.037586:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13571 00000100:00100000:3.0:1769074848.037595:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:66757:x1855007617028224:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074848.037619:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074848.037639:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29200/0x7fe2323adfd55750 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c832 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.037692:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29200/0x7fe2323adfd55750 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c832 expref: 19 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.037705:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29200/0x7fe2323adfd55750 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c832 expref: 19 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.037723:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074848.037726:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074848.037729:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29200, rc 0) 00000100:00100000:3.0:1769074848.037778:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66757:x1855007617028224:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 183us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074848.037785:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13571 00000100:00100000:1.0:1769074848.040607:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625235328:12345-0@lo:2:osp-syn-0-0.0 Request processed in 6736us (8411us total) trans 21474841303 rc 0/0 00000100:00100000:1.0:1769074848.040622:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14516 00000004:00080000:1.0:1769074848.040700:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d1dd880/2, rc 0, transno 21474841304 00000100:00100000:1.0:1769074848.040708:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625235200:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1769074848.041174:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d1dd180/2, rc 0, transno 21474841303 00000100:00100000:3.0:1769074848.041181:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625235328:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074848.042402:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074848.042494:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617028480 00000100:00100000:0.0:1769074848.042503:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617028480 00000100:00100000:0.0:1769074848.042559:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4741 00000100:00100000:0.0:1769074848.042569:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:66757:x1855007617028480:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074848.042840:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66757:x1855007617028480:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 272us (455us total) trans 8589944116 rc 0/0 00000100:00100000:0.0:1769074848.042849:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4741 00000100:00100000:2.0:1769074848.066372:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074848.066461:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617028736 00000100:00100000:0.0:1769074848.066471:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617028736 00000100:00100000:0.0:1769074848.066532:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13572 00000100:00100000:0.0:1769074848.066542:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66758:x1855007617028736:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074848.066569:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074848.066591:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55757 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c839 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074848.066663:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066676:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066687:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066699:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066706:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066711:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066784:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066792:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066798:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066848:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066854:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074848.066858:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066876:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066888:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80000) 00010000:00010000:0.0:1769074848.066890:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074848.066895:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066900:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd5575e lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066913:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066919:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066924:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066929:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074848.066932:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066938:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066949:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b643a80500) 00010000:00010000:0.0:1769074848.066952:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074848.066954:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.066959:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd55765 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074848.067243:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55757 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c839 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074848.067255:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b643a80000) 00010000:00010000:0.0:1769074848.067257:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074848.067260:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134400/0x7fe2323adfd55757 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c839 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074848.067272:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134400, rc 0) 00000100:00100000:2.0:1769074848.067676:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66758:x1855007617028736:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1132us (1311us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074848.067685:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13572 00000100:00100000:3.0:1769074848.073950:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.074148:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617028864 00000100:00100000:2.0:1769074848.074159:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617028864 00000100:00100000:2.0:1769074848.074230:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13573 00000100:00100000:2.0:1769074848.074241:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66758:x1855007617028864:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074848.074318:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074336:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074348:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074361:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074368:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074374:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074459:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074468:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074475:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074848.074510:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:2.0:1769074848.074514:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074848.074527:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074541:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074547:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074553:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074559:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074569:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.074573:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074580:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074592:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074848.074595:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.074600:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074606:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd5576c lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074623:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074630:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074635:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074640:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.074643:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074649:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074655:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effac00) 00010000:00010000:2.0:1769074848.074658:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.074661:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074666:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55773 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074676:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074683:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074688:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074692:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.074694:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074699:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074705:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.074708:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.074710:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.074715:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5577a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074848.074796:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66758:x1855007617028864:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 557us (854us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.074835:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13573 00000100:00100000:0.0:1769074848.106719:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.106846:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617029120 00000100:00100000:3.0:1769074848.106857:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617029120 00000100:00100000:3.0:1769074848.106923:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13574 00000100:00100000:3.0:1769074848.106932:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b66e720c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66762:x1855007617029120:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074848.106953:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074848.106977:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55781 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c840 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.107064:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107078:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107090:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107101:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107107:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107113:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107192:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107199:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107219:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107226:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107232:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.107236:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107244:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107256:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.107259:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.107264:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107271:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55788 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107286:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107292:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107297:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107301:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.107304:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107309:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107316:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea800) 00010000:00010000:3.0:1769074848.107319:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.107321:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107327:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5578f lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.107359:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55781 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c840 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.107369:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.107371:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074848.107375:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55781 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c840 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.107402:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29800, rc 0) 00000100:00100000:3.0:1769074848.107456:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b66e720c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66762:x1855007617029120:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 525us (747us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074848.107464:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13574 00000100:00100000:0.0:1769074848.114551:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.114663:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617029248 00000100:00100000:2.0:1769074848.114673:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617029248 00000100:00100000:2.0:1769074848.114738:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13575 00000100:00100000:2.0:1769074848.114749:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66762:x1855007617029248:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074848.114882:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.114899:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.114912:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.114933:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.114941:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.114947:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115021:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115029:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115036:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074848.115086:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:2.0:1769074848.115091:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074848.115104:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115121:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115127:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115133:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115140:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115145:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.115149:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115156:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115167:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074848.115170:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.115174:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115181:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55796 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115199:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115205:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115210:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115215:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.115217:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115223:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115229:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa600) 00010000:00010000:2.0:1769074848.115232:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.115234:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115239:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5579d lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115249:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115255:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115259:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115263:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.115266:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115270:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115276:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.115279:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.115281:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.115286:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557a4 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074848.115356:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66762:x1855007617029248:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 608us (815us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.115363:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13575 00010000:00010000:1.0:1769074848.157159:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074848.164734:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.164852:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617029504 00000100:00100000:3.0:1769074848.164863:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617029504 00000100:00100000:3.0:1769074848.164922:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13576 00000100:00100000:3.0:1769074848.164932:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66770:x1855007617029504:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074848.164954:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074848.164977:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd557ab lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c847 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.165063:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165077:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165090:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165102:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165108:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165113:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165190:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165197:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165202:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165208:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165214:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.165218:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165226:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165236:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.165239:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.165243:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165248:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557b2 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165260:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165266:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165271:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165277:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.165280:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165285:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165291:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea800) 00010000:00010000:3.0:1769074848.165294:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.165296:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165301:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557b9 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.165333:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd557ab lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c847 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.165343:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.165346:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074848.165349:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd557ab lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c847 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.165360:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29800, rc 0) 00000100:00100000:3.0:1769074848.165419:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66770:x1855007617029504:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 489us (691us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074848.165426:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13576 00000100:00100000:0.0:1769074848.168910:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.168988:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617029632 00000100:00100000:2.0:1769074848.168998:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617029632 00000100:00100000:2.0:1769074848.169063:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13577 00000100:00100000:2.0:1769074848.169073:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66770:x1855007617029632:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074848.169155:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169174:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169186:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169199:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169206:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169212:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169286:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169295:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169302:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074848.169343:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:2.0:1769074848.169347:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074848.169360:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169374:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169380:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169386:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169392:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169398:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.169413:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169420:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169432:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074848.169435:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.169652:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169661:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557c0 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169681:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169686:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169692:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169697:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.169700:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169705:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169712:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effac00) 00010000:00010000:2.0:1769074848.169715:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.169717:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169722:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557c7 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169731:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169736:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169741:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169747:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.169749:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169754:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169760:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.169762:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.169765:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.169769:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ce lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074848.170150:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66770:x1855007617029632:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1076us (1249us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.170159:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13577 00000100:00100000:0.0:1769074848.211679:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.211775:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617029888 00000100:00100000:3.0:1769074848.211787:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617029888 00000100:00100000:3.0:1769074848.211894:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13578 00000100:00100000:3.0:1769074848.211905:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66777:x1855007617029888:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074848.211927:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074848.211952:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd557d5 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c84e expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.212051:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212068:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212080:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212094:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212100:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212107:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212193:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212201:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212207:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212213:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212220:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.212224:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212231:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212244:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.212247:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.212251:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212257:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd557dc lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212271:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212276:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212281:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212286:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.212290:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212295:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212301:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea800) 00010000:00010000:3.0:1769074848.212303:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.212306:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212311:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd557e3 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.212344:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd557d5 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c84e expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.212354:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.212356:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074848.212360:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd557d5 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c84e expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.212371:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29800, rc 0) 00000100:00100000:3.0:1769074848.212447:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66777:x1855007617029888:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 534us (773us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074848.212456:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13578 00000100:00100000:0.0:1769074848.220229:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.220332:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617030016 00000100:00100000:2.0:1769074848.220344:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617030016 00000100:00100000:2.0:1769074848.220411:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13579 00000100:00100000:2.0:1769074848.220421:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66777:x1855007617030016:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074848.220510:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220527:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220539:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220555:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220562:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220569:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220643:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220652:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220659:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074848.220701:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:2.0:1769074848.220705:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074848.220718:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220733:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220739:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220745:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220752:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220762:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.220766:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220773:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220785:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074848.220788:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.220793:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.220799:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd557ea lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221104:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221112:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221117:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221122:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.221125:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221130:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221138:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa600) 00010000:00010000:2.0:1769074848.221140:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.221143:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221148:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd557f1 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221161:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221166:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221171:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221175:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.221178:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221182:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221188:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.221190:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.221192:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.221197:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557f8 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074848.221270:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66777:x1855007617030016:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 848us (1051us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.221279:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13579 00000100:00100000:0.0:1769074848.270719:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.270857:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617030272 00000100:00100000:2.0:1769074848.270878:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617030272 00000100:00100000:2.0:1769074848.270938:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13580 00000100:00100000:2.0:1769074848.270947:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66785:x1855007617030272:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074848.270967:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074848.270990:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557ff lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c855 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.271071:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271085:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271095:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271107:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271113:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271119:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271186:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271193:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271198:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271208:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271214:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.271218:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271225:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271237:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074848.271240:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.271245:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271250:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55806 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271263:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271269:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271275:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271280:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.271282:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271288:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271299:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effac00) 00010000:00010000:2.0:1769074848.271301:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.271303:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271308:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5580d lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.271337:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557ff lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c855 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.271346:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074848.271349:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074848.271352:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd557ff lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c855 expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.271362:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:3.0:1769074848.271748:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66785:x1855007617030272:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 801us (1038us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074848.271757:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13580 00000100:00100000:0.0:1769074848.279344:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.279527:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617030400 00000100:00100000:3.0:1769074848.279540:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617030400 00000100:00100000:3.0:1769074848.279606:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13581 00000100:00100000:3.0:1769074848.279617:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66785:x1855007617030400:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074848.279708:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279727:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279743:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279756:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279763:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279769:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279909:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279919:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.279925:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074848.279967:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:3.0:1769074848.279972:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074848.279985:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280001:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280007:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280013:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280023:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280028:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.280032:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280051:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280064:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.280067:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.280072:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280078:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55814 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280095:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280101:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280106:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280111:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.280114:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280119:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280130:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea800) 00010000:00010000:3.0:1769074848.280132:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.280135:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280140:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5581b lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280150:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280156:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280162:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280166:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.280169:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280174:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280181:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074848.280183:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.280186:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.280191:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55822 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074848.280275:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66785:x1855007617030400:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 659us (940us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074848.280284:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13581 00000100:00100000:0.0:1769074848.326159:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.326351:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617030656 00000100:00100000:3.0:1769074848.326363:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617030656 00000100:00100000:3.0:1769074848.326433:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13582 00000100:00100000:3.0:1769074848.326451:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66791:x1855007617030656:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074848.326472:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074848.326494:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55829 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c85c expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.326569:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326581:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326591:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326602:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326608:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326614:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326683:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326690:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326695:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326701:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326706:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.326709:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326715:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326727:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:3.0:1769074848.326730:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.326734:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326740:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55830 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326751:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326757:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326762:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326767:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.326770:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326781:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326788:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea700) 00010000:00010000:3.0:1769074848.326790:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.326792:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326798:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55837 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.326893:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55829 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c85c expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.326902:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:3.0:1769074848.326904:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074848.326908:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55829 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c85c expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.326917:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29400, rc 0) 00000100:00100000:3.0:1769074848.326971:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66791:x1855007617030656:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 522us (823us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074848.326978:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13582 00000100:00100000:2.0:1769074848.331750:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.331868:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617030784 00000100:00100000:3.0:1769074848.331879:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617030784 00000100:00100000:3.0:1769074848.331940:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13583 00000100:00100000:3.0:1769074848.331950:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66791:x1855007617030784:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074848.332032:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332059:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332071:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332084:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332091:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332097:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332170:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332180:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332186:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074848.332410:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:3.0:1769074848.332415:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074848.332429:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332453:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332460:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332470:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332476:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332481:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.332485:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332493:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332506:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea200) 00010000:00010000:3.0:1769074848.332509:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.332513:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332518:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd5583e lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332536:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332541:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332546:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332550:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.332553:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332562:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332568:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea000) 00010000:00010000:3.0:1769074848.332571:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.332572:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332577:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55845 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332586:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332590:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332594:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332598:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.332601:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332606:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332612:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074848.332615:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.332617:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074848.332622:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5584c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074848.332769:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66791:x1855007617030784:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 818us (1028us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074848.332777:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13583 00000100:00100000:0.0:1769074848.380220:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.380549:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031040 00000100:00100000:3.0:1769074848.380562:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031040 00000100:00100000:3.0:1769074848.380631:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13584 00000100:00100000:3.0:1769074848.380642:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66801:x1855007617031040:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074848.380664:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074848.380693:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55853 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c863 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.380776:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380792:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380844:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380870:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380877:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380884:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380978:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380986:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380993:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.380999:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381005:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.381009:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381017:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381029:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea000) 00010000:00010000:3.0:1769074848.381032:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.381046:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381053:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd5585a lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381067:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381073:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381079:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381083:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.381088:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381093:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381100:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea200) 00010000:00010000:3.0:1769074848.381103:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.381105:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381111:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55861 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.381147:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55853 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c863 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.381158:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea000) 00010000:00010000:3.0:1769074848.381160:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074848.381163:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55853 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c863 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074848.381174:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28000, rc 0) 00000100:00100000:3.0:1769074848.381249:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66801:x1855007617031040:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 608us (1038us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074848.381259:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13584 00000100:00100000:1.0:1769074848.399107:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.399215:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031168 00000100:00100000:2.0:1769074848.399227:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031168 00000100:00100000:2.0:1769074848.399298:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13585 00000100:00100000:2.0:1769074848.399308:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66801:x1855007617031168:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074848.399401:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399419:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399433:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399450:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399456:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399463:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399536:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399544:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399552:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074848.399590:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:2.0:1769074848.399595:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074848.399608:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399620:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399627:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399633:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399639:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399650:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.399654:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399662:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399673:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074848.399676:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.399681:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399686:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55868 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399704:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399710:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399716:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399720:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.399722:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399728:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399734:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effac00) 00010000:00010000:2.0:1769074848.399736:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.399739:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399744:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5586f lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399754:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399760:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399765:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399769:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.399772:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399778:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399784:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074848.399786:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.399789:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.399795:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55876 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074848.400243:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66801:x1855007617031168:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 935us (1143us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.400252:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13585 00000100:00100000:0.0:1769074848.428485:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074848.428622:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031424 00000100:00100000:2.0:1769074848.428634:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031424 00000100:00100000:2.0:1769074848.428698:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13586 00000100:00100000:2.0:1769074848.428708:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542faad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66802:x1855007617031424:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074848.428729:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074848.428754:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5587d lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c86a expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.428885:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.428900:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.428911:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.428923:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.428930:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.428935:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429002:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429009:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429015:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429025:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429031:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.429035:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429053:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429065:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074848.429068:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.429073:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429078:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55884 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429092:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429097:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429103:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429107:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074848.429110:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429115:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429126:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa600) 00010000:00010000:2.0:1769074848.429129:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074848.429131:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429136:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd5588b lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074848.429169:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5587d lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c86a expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.429179:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effac00) 00010000:00010000:2.0:1769074848.429182:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074848.429185:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5587d lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c86a expref: 21 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074848.429197:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89200, rc 0) 00000100:00100000:2.0:1769074848.429268:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542faad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66802:x1855007617031424:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 562us (794us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074848.429276:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13586 00000100:00100000:0.0:1769074848.434060:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074848.434181:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031552 00000100:00100000:3.0:1769074848.434191:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031552 00000100:00100000:3.0:1769074848.434250:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13587 00000100:00100000:3.0:1769074848.434261:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66802:x1855007617031552:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074848.434342:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.434358:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.434370:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.434736:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.434746:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.434753:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435016:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435026:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435033:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074848.435086:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:3.0:1769074848.435090:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074848.435103:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435117:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435127:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435133:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435139:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435145:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.435149:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435156:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435167:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b6543ea000) 00010000:00010000:3.0:1769074848.435170:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.435175:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435180:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd55892 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435196:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435202:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435207:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435212:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.435215:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435221:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435227:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b6543ea200) 00010000:00010000:3.0:1769074848.435229:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.435232:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435238:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29a00/0x7fe2323adfd55899 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435248:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435254:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435259:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435263:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074848.435266:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435270:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435277:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074848.435279:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074848.435282:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074848.435289:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd558a0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074848.435428:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:66802:x1855007617031552:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1166us (1385us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074848.435436:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13587 00000100:00100000:1.0:1769074848.669246:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625235712:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669307:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.669342:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625236224:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669360:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.669371:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625236736:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669389:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.669412:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:1.0:1769074848.669417:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625235584:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669433:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.669445:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625236096:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669459:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.669473:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625236608:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669486:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074848.669519:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235712 00000100:00100000:1.0:1769074848.669527:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235712 00000100:00100000:1.0:1769074848.669569:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236224 00000100:00100000:1.0:1769074848.669571:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236224 00000100:00100000:1.0:1769074848.669592:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236096 00000100:00100000:1.0:1769074848.669593:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236096 00000100:00100000:1.0:1769074848.669607:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2439 00000100:00100000:1.0:1769074848.669613:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625235712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669659:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625235712:12345-0@lo:400:kworker.0 Request processed in 46us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074848.669666:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2439 00000100:00100000:1.0:1769074848.669681:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2440 00000100:00100000:1.0:1769074848.669686:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625236224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669706:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625236224:12345-0@lo:400:kworker.0 Request processed in 21us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074848.669711:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2440 00000100:00100000:1.0:1769074848.669721:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2441 00000100:00100000:1.0:1769074848.669725:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625236096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669745:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625236096:12345-0@lo:400:kworker.0 Request processed in 19us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074848.669751:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2441 00000100:00080000:1.0:1769074848.669765:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074848.669768:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074848.669794:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625236096:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669832:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625235712:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669842:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a272d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625236224:0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669869:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236736 00000100:00100000:1.0:1769074848.669872:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236736 00000100:00100000:1.0:1769074848.669892:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1535 00000100:00100000:1.0:1769074848.669897:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625236736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.669926:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+9:3652:x1855007625236736:12345-0@lo:400:kworker.0 Request processed in 29us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074848.669931:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1535 00000100:00100000:1.0:1769074848.669950:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625236736:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671117:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625235456:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671181:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671246:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625235968:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671272:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671291:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625236480:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671310:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671323:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625236992:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671339:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671357:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1769074848.671361:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625235840:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671376:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671389:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625236352:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671403:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671411:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625236864:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671426:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074848.671454:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236480 00000100:00100000:2.0:1769074848.671460:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236480 00000100:00100000:2.0:1769074848.671496:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236992 00000100:00100000:2.0:1769074848.671498:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236992 00000100:00100000:2.0:1769074848.671507:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236352 00000100:00100000:2.0:1769074848.671509:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236352 00000100:00100000:2.0:1769074848.671521:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2442 00000100:00100000:2.0:1769074848.671526:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625236480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671567:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625236480:12345-0@lo:400:kworker.0 Request processed in 40us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.671572:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2442 00000100:00100000:2.0:1769074848.671587:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2443 00000100:00100000:2.0:1769074848.671592:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625236992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671617:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625236992:12345-0@lo:400:kworker.0 Request processed in 26us (277us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.671622:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2443 00000100:00100000:2.0:1769074848.671632:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2444 00000100:00100000:2.0:1769074848.671637:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625236352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671658:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625236352:12345-0@lo:400:kworker.0 Request processed in 22us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.671663:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2444 00000100:00100000:2.0:1769074848.671691:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625236352:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671699:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625236480:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671707:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625236992:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671732:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235968 00000100:00100000:2.0:1769074848.671735:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235968 00000100:00100000:2.0:1769074848.671757:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235840 00000100:00100000:2.0:1769074848.671759:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235840 00000100:00100000:2.0:1769074848.671775:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236864 00000100:00100000:2.0:1769074848.671777:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236864 00000100:00100000:2.0:1769074848.671789:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1536 00000100:00100000:2.0:1769074848.671793:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625235968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671859:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625235968:12345-0@lo:400:kworker.0 Request processed in 66us (586us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.671864:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1536 00000100:00100000:2.0:1769074848.671874:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1537 00000100:00100000:2.0:1769074848.671879:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625235840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671902:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625235840:12345-0@lo:400:kworker.0 Request processed in 23us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.671907:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1537 00000100:00100000:2.0:1769074848.671916:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1538 00000100:00100000:2.0:1769074848.671921:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625236864:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671939:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625236864:12345-0@lo:400:kworker.0 Request processed in 19us (512us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074848.671943:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1538 00000100:00100000:2.0:1769074848.671967:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625235840:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671975:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625236864:0@lo:400:kworker.0 00000100:00100000:2.0:1769074848.671982:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 17179874623 gen 2 00000004:00080000:2.0:1769074848.671988:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f2b800, transno 17179874623 00000100:00100000:2.0:1769074848.671997:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625235968:0@lo:400:kworker.0 00000100:00100000:0.0:1769074848.672337:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235584 00000100:00100000:0.0:1769074848.672343:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235584 00000100:00100000:0.0:1769074848.672380:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625236608 00000100:00100000:0.0:1769074848.672383:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625236608 00000100:00100000:0.0:1769074848.672400:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2718 00000100:00100000:0.0:1769074848.672406:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625235584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074848.672453:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625235584:12345-0@lo:400:kworker.0 Request processed in 47us (3018us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074848.672458:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2718 00000100:00100000:0.0:1769074848.672471:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2719 00000100:00100000:0.0:1769074848.672476:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625236608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074848.672502:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625236608:12345-0@lo:400:kworker.0 Request processed in 26us (3015us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074848.672506:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2719 00000100:00100000:0.0:1769074848.672541:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625235584:0@lo:400:kworker.0 00000100:00100000:0.0:1769074848.672550:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968368 gen 2 00000004:00080000:0.0:1769074848.672555:0:3652:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968367 committed transno 4294968368 00000020:00080000:0.0:1769074848.672561:0:3652:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff89b64ef5b240 refcount 2 committed 0 result 0 batchid 4294969367 00000020:00080000:0.0:1769074848.672565:0:3652:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b5449a0a00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b64ef5b180 00000020:00080000:0.0:1769074848.672569:0:3652:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000405:0x2:0x0].7 00000020:00080000:0.0:1769074848.672571:0:3652:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b5449a0200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff89b545bbc600 00000020:00080000:0.0:1769074848.672574:0:3652:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].85 00000004:00080000:0.0:1769074848.672592:0:3652:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968368 committed transno 4294968368 00000100:00100000:0.0:1769074848.672603:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625236608:0@lo:400:kworker.0 00000100:00080000:0.0:1769074848.672636:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074848.672642:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074848.672645:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074848.672689:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625235456 00000100:00100000:1.0:1769074848.672694:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625235456 00000100:00100000:1.0:1769074848.672731:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1769074848.672737:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b649912450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625235456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074848.672773:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b649912450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625235456:12345-0@lo:400:kworker.0 Request processed in 37us (1592us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074848.672778:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1769074848.672838:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625235456:0@lo:400:kworker.0 00000020:00080000:3.0:1769074848.673017:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294969367 committed batchid 4294969366 00000020:00080000:3.0:1769074848.673058:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969366 00010000:00010000:2.0:1769074849.180990:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074849.734921:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074849.734994:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074849.735041:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074849.735069:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074849.735091:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074849.735137:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031680 00000100:00100000:1.0:1769074849.735147:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031680 00000100:00100000:1.0:1769074849.735199:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 373 00000100:00100000:1.0:1769074849.735211:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f81c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617031680:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074849.735286:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f81c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617031680:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 76us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074849.735295:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 373 00000100:00100000:1.0:1769074849.735350:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031808 00000100:00100000:1.0:1769074849.735354:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031808 00000100:00100000:1.0:1769074849.735392:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617031936 00000100:00100000:1.0:1769074849.735395:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617031936 00000100:00100000:1.0:1769074849.735417:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2445 00000100:00100000:1.0:1769074849.735424:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007617031808:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074849.735455:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2406:x1855007617031808:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074849.735460:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2445 00000100:00100000:1.0:1769074849.735472:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2446 00000100:00100000:1.0:1769074849.735476:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2406:x1855007617031936:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074849.735496:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2406:x1855007617031936:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 20us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074849.735516:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2446 00000100:00100000:1.0:1769074849.735536:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032064 00000100:00100000:1.0:1769074849.735539:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032064 00000100:00100000:1.0:1769074849.735579:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032192 00000100:00100000:1.0:1769074849.735583:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032192 00000100:00100000:1.0:1769074849.735608:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1539 00000100:00100000:1.0:1769074849.735616:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617032064:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074849.735739:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617032064:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 123us (743us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074849.735746:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1539 00000100:00100000:1.0:1769074849.735763:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1540 00000100:00100000:1.0:1769074849.735771:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2406:x1855007617032192:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074849.735797:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2406:x1855007617032192:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 27us (705us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074849.735803:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1540 00000100:00080000:1.0:1769074849.735951:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074849.735956:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074849.735958:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074849.735959:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1769074850.205202:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074850.717025:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1769074850.717050:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625237248:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074850.717153:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074850.717198:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625237120:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074850.717215:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237248 00000100:00100000:2.0:1769074850.717220:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074850.717223:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237248 00000100:00100000:3.0:1769074850.717278:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237120 00000100:00100000:3.0:1769074850.717282:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237120 00000100:00100000:3.0:1769074850.717303:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2720 00000100:00100000:3.0:1769074850.717310:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625237248:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074850.717344:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445380096 avail 1299140608 00000100:00100000:3.0:1769074850.717390:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625237248:12345-0@lo:41:osp-pre-0-1.0 Request processed in 79us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074850.717396:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2720 00000100:00100000:3.0:1769074850.717414:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2721 00000100:00100000:3.0:1769074850.717420:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625237120:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074850.717435:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446137856 avail 1299898368 00000100:00100000:3.0:1769074850.717482:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625237120:12345-0@lo:41:osp-pre-1-0.0 Request processed in 62us (259us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074850.717488:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2721 00000100:00100000:2.0:1769074850.717727:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625237248:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074850.717738:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625237120:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074851.229269:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625237376:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074851.229366:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074851.229666:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237376 00000100:00100000:2.0:1769074851.229674:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237376 00000100:00100000:2.0:1769074851.229724:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1228 00000100:00100000:2.0:1769074851.229733:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625237376:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074851.229763:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 1073152 granted 29742400 pending 0 free 3914907648 avail 3683385344 00000100:00100000:2.0:1769074851.229845:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625237376:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074851.229856:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1228 00000100:00100000:2.0:1769074851.229906:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625237376:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1769074851.230875:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1769074851.230885:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074851.230911:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625237504:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074851.230955:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074851.232454:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237504 00000100:00100000:0.0:1769074851.232464:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237504 00000100:00100000:0.0:1769074851.232550:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1229 00000100:00100000:0.0:1769074851.232561:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625237504:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074851.232589:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3912896512 avail 3681374208 00000100:00100000:0.0:1769074851.232656:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625237504:12345-0@lo:13:osp-pre-0-1.0 Request processed in 98us (1699us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074851.232664:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1229 00000100:00080000:0.0:1769074851.232703:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074851.232742:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625237504:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074851.232761:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074851.232769:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625237632:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074851.232800:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074851.233034:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237632 00000100:00100000:0.0:1769074851.233039:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237632 00000100:00100000:0.0:1769074851.233067:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1230 00000100:00100000:0.0:1769074851.233074:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625237632:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074851.233085:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3912896512 avail 3681374208 00000100:00100000:0.0:1769074851.233116:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3649:x1855007625237632:12345-0@lo:13:osp-pre-0-0.0 Request processed in 42us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074851.233121:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1230 00000100:00080000:0.0:1769074851.233137:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074851.233155:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625237632:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074851.741091:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074851.741118:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625237760:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074851.741191:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074851.741486:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237760 00000100:00100000:0.0:1769074851.741496:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237760 00000100:00100000:0.0:1769074851.741575:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1231 00000100:00100000:0.0:1769074851.741584:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625237760:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074851.741611:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 1073152 granted 29742400 pending 0 free 3914907648 avail 3683385344 00000100:00080000:2.0:1769074851.741712:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074851.743127:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625237760:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1537us (1929us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074851.743140:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1231 00000100:00100000:3.0:1769074851.745612:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625237760:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074852.253081:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074853.291028:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1769074853.305006:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841304 is committed 00000001:00080000:1.0:1769074853.305025:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841302 is committed 00000100:00100000:1.0:1769074853.479131:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074853.482477:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032320 00000100:00100000:2.0:1769074853.482494:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032320 00000100:00100000:2.0:1769074853.482589:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13588 00000100:00100000:2.0:1769074853.482601:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:67226:x1855007617032320:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:2.0:1769074853.482622:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074853.482646:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558a7 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c871 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.482716:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.482731:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.482755:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.482832:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.482840:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.482846:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483243:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 3/1,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483254:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483261:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074853.483305:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x207:0x0]: returning size 2097152 00000004:00200000:2.0:1769074853.483310:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074853.483329:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483341:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483349:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483354:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483360:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483366:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483372:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074853.483376:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483382:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483393:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa600) 00010000:00010000:2.0:1769074853.483396:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074853.483401:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483407:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd558ae lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483421:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483427:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483433:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483438:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074853.483441:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483446:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483453:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effaa00) 00010000:00010000:2.0:1769074853.483455:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074853.483457:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483462:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd558b5 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074853.483496:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558a7 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c871 expref: 22 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.483514:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c871 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.483521:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074853.483524:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074853.483526:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074853.483599:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67226:x1855007617032320:12345-192.168.204.60@tcp:101:rm.0 Request processed in 1001us (4478us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074853.483607:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13588 00000100:00080000:0.0:1769074853.486058:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074853.488629:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074853.488692:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032576 00000100:00100000:2.0:1769074853.488702:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032576 00000100:00100000:2.0:1769074853.488761:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13589 00000100:00100000:2.0:1769074853.488771:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67226:x1855007617032576:12345-192.168.204.60@tcp:36:rm.0 00010000:00010000:2.0:1769074853.488868:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1769074853.488895:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29200/0x7fe2323adfd55750 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x207:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c832 expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.488914:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd558bc lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c871 expref: 20 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.488924:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55574 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x207:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c79f expref: 19 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.488933:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074853.488942:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074853.488980:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 3/0,1 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.488994:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489004:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489016:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489022:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489028:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489103:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489110:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489116:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000040:00080000:2.0:1769074853.489348:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2469, 64 off210960 00000004:00080000:2.0:1769074853.489359:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1841331551329ns, next at 1842331548623ns, rc = 0 00000040:00080000:2.0:1769074853.489372:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2470, 64 off211024 00000040:00080000:2.0:1769074853.489394:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2473, 64 off211552 00000004:00080000:2.0:1769074853.489400:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1841331593063ns, next at 1842331591778ns, rc = 0 00000040:00080000:2.0:1769074853.489411:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2474, 64 off211616 00010000:00010000:2.0:1769074853.489471:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489484:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489492:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489499:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489505:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489511:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489523:0:6514:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1769074853.489541:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489547:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489553:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489559:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074853.489563:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489570:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489579:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effaf00) 00010000:00010000:2.0:1769074853.489582:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1769074853.489585:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489591:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd558d1 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x207:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489728:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489735:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489741:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 3/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489746:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489751:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074853.489753:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489759:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 5/0,0 mode: CW/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489767:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0x0 (ffff89b64effa500) 00010000:00010000:2.0:1769074853.489770:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074853.489773:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 3/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489778:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd558c3 lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000004:00010000:2.0:1769074853.489791:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000f6a641f7 reply state 000000001be3a913 transno 8589944117 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489830:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.489837:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa700) 00010000:00010000:2.0:1769074853.489839:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074853.489842:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074853.489913:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:67226:x1855007617032576:12345-192.168.204.60@tcp:36:rm.0 Request processed in 1142us (1290us total) trans 8589944117 rc 0/0 00000100:00100000:2.0:1769074853.489921:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13589 00010000:00010000:1.0:1769074853.490293:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074853.490299:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074853.490306:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074853.490309:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074853.490315:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074853.490324:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x207:0x0].0x0 (ffff89b64effaf00) 00010000:00010000:1.0:1769074853.490326:0:6502:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1769074853.490330:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 1/0,0 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074853.490335:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd558d8 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x207:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074853.490349:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074853.490350:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074853.490352:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1769074853.511122:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944117 is committed 00000004:00080000:3.0:1769074853.511141:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1969 committed 00000004:00080000:3.0:1769074853.511153:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1967 committed 00000020:00080000:3.0:1769074853.511177:0:32377:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 4294969367 batchid updated 00010000:00010000:2.0:1769074853.511443:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.511456:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.511464:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.511471:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074853.511476:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.511485:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.511503:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x8:0x0].0xe563034c (ffff89b64effa700) 00010000:00010000:2.0:1769074853.511506:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074853.511512:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074853.511518:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd558ca lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x8:0x0].0xe563034c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:2.0:1769074853.511748:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625238016:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1769074853.511836:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074853.511937:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625238016 00000100:00100000:2.0:1769074853.511946:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625238016 00000100:00100000:2.0:1769074853.512009:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14518 00000100:00100000:2.0:1769074853.512017:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625238016:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1769074853.512043:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:711 count 1 00010000:00010000:2.0:1769074853.512065:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.512083:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.512090:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074853.512092:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074853.512096:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.512119:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.512133:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 9 pid: 27843 timeout: 1941 lvb_type: 0 00000100:00100000:2.0:1769074853.512143:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625238144:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074853.512198:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5410b8240 going to sleep for 11 seconds 00000100:00080000:2.0:1769074853.512222:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074853.514664:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074853.514689:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a270e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625237888:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1769074853.514753:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.514794:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625237888 00000100:00100000:1.0:1769074853.514801:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625237888 00000100:00100000:1.0:1769074853.514891:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14519 00000100:00100000:1.0:1769074853.514900:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625237888:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1769074853.514927:0:28955:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:713 count 1 00010000:00010000:1.0:1769074853.514965:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074853.514986:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074853.514994:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074853.514997:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074853.515000:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074853.515030:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 10 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074853.515044:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 10 pid: 36516 timeout: 1941 lvb_type: 0 00000100:00100000:1.0:1769074853.515057:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625238272:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074853.515126:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7fcc0 going to sleep for 11 seconds 00000100:00080000:1.0:1769074853.515138:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000020:00080000:2.0:1769074853.516254:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969367 00000004:00080000:2.0:1769074853.516321:0:32406:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b5410b8a80 with version 20 00000020:00080000:2.0:1769074853.516340:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294969367 cancel update log [0x240000405:0x2:0x0].7: rc = 0 00000020:00080000:2.0:1769074853.516425:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294969367 cancel update log [0x200000bd0:0x2:0x0].85: rc = 0 00000020:00080000:2.0:1769074853.516438:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969367 00000004:00080000:1.0:1769074853.516594:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 20 rpc_version 20 00000020:00100000:1.0:1769074853.516623:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1769074853.516632:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b67a271500 x1855007625238528/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1769074853.516659:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625238528:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074853.516714:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.516755:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769074853.516982:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625238528 00000100:00100000:2.0:1769074853.516990:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625238528 00000100:00100000:2.0:1769074853.517038:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2722 00000100:00100000:2.0:1769074853.517045:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625238528:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1769074853.517350:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625238528:12345-0@lo:1000:osp_up1-0.0 Request processed in 304us (636us total) trans 4294968369 rc 0/0 00000100:00100000:2.0:1769074853.517359:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2722 00000100:00100000:1.0:1769074853.517560:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a271500 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625238528:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074853.524834:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074853.524877:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074853.536622:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a273100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625238272:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074853.536660:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074853.536712:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032704 00000100:00100000:1.0:1769074853.536718:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032704 00010000:00010000:1.0:1769074853.536790:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 10 pid: 27843 timeout: 1941 lvb_type: 0 00000100:00080000:2.0:1769074853.536841:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074853.537997:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032832 00000100:00100000:1.0:1769074853.538006:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032832 00010000:00010000:1.0:1769074853.538043:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 11 pid: 36516 timeout: 1941 lvb_type: 0 00000100:00100000:1.0:1769074853.538071:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7220 00000100:00100000:1.0:1769074853.538081:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617032704:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074853.538095:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074853.538101:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 10 pid: 27843 timeout: 1941 lvb_type: 0 00010000:00010000:1.0:1769074853.538119:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 10 pid: 27843 timeout: 1941 lvb_type: 0 00010000:00010000:1.0:1769074853.538143:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074853.538145:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074853.538150:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074853.538160:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074853.538179:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074853.539028:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549134200/0x7fe2323adfd5573b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c824 expref: 10 pid: 27843 timeout: 1941 lvb_type: 0 00000100:00100000:3.0:1769074853.539045:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625238144:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074853.539066:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074853.539077:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074853.539084:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074853.539094:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074853.539105:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7221 00010000:00010000:3.0:1769074853.539117:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074853.539122:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617032832:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074853.539137:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074853.539142:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1769074853.539144:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 11 pid: 36516 timeout: 1941 lvb_type: 0 00010000:00010000:0.0:1769074853.539157:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 11 pid: 36516 timeout: 1941 lvb_type: 0 00010000:00010000:0.0:1769074853.539175:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549242a00/0x7fe2323adfd55726 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c816 expref: 11 pid: 36516 timeout: 1941 lvb_type: 0 00010000:00010000:0.0:1769074853.539187:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074853.539189:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074853.539192:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074853.539449:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617032704:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1367us (14648us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.539457:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7220 00010000:00010000:0.0:1769074853.540507:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074853.540555:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074853.540600:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617032832:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 1481us (15720us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074853.540609:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7221 00010000:00010000:0.0:1769074853.540729:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074853.540737:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074853.540747:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074853.540753:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074853.540782:0:28955:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1769074853.541147:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1769074853.541157:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541166:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074853.541170:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541180:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541198:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541219:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd558df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541238:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074853.541241:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541247:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074853.541249:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541257:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541267:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541274:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549242e00/0x7fe2323adfd558e6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074853.541287:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074853.541289:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074853.541292:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074853.541471:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1769074853.541480:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074853.554632:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3651:x1855007625237888:12345-0@lo:6:osp-syn-1-0.0 Request processed in 39725us (39870us total) trans 17179874632 rc 0/0 00000100:00100000:0.0:1769074853.554648:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14519 00000004:00080000:0.0:1769074853.554714:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a270e00/2, rc 0, transno 17179874632 00000100:00100000:0.0:1769074853.554726:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a270e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625237888:0@lo:6:osp-syn-1-0.0 00000001:00080000:0.0:1769074853.558574:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874632 is committed 00000001:00080000:0.0:1769074853.558594:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874629 is committed 00000100:00100000:3.0:1769074853.566080:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625238016:12345-0@lo:6:osp-syn-0-0.0 Request processed in 54055us (54234us total) trans 21474841305 rc 0/0 00000100:00100000:3.0:1769074853.566109:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14518 00000100:00100000:3.0:1769074853.566184:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841304 gen 2 00000004:00080000:3.0:1769074853.566191:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547599180, transno 21474841299 00000004:00080000:3.0:1769074853.566201:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547598e00, transno 21474841300 00000004:00080000:3.0:1769074853.566204:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d1dd180, transno 21474841303 00000004:00080000:3.0:1769074853.566207:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d1dd880, transno 21474841304 00000100:00100000:3.0:1769074853.566214:0:3649:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b5461e6a00 x1855007625238016/t21474841305(21474841305) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1769074869 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:3.0:1769074853.566227:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5461e6a00/2, rc 0, transno 21474841305 00000100:00100000:3.0:1769074853.566232:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625238016:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1769074853.566371:0:32394:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 1 changes, 1 in progress, 0 in flight, 4 done 00000100:00100000:1.0:1769074853.790153:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625238784:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790226:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.790268:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625239296:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790292:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.790311:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625239808:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790331:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.790354:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1769074853.790360:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625238912:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790380:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.790397:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625239424:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790428:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074853.790460:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239296 00000100:00100000:1.0:1769074853.790467:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239296 00000100:00100000:1.0:1769074853.790518:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239808 00000100:00100000:1.0:1769074853.790522:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239808 00000100:00100000:1.0:1769074853.790533:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625238912 00000100:00100000:1.0:1769074853.790536:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625238912 00000100:00100000:1.0:1769074853.790548:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2447 00000100:00100000:1.0:1769074853.790569:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625239296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790631:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625239296:12345-0@lo:400:kworker.0 Request processed in 52us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.790638:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2447 00000100:00100000:1.0:1769074853.790654:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239424 00000100:00100000:1.0:1769074853.790658:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239424 00000100:00100000:1.0:1769074853.790673:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2448 00000100:00100000:1.0:1769074853.790679:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625239808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790705:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625239808:12345-0@lo:400:kworker.0 Request processed in 26us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.790710:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2448 00000100:00100000:1.0:1769074853.790721:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2449 00000100:00100000:1.0:1769074853.790727:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625238912:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790750:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625238912:12345-0@lo:400:kworker.0 Request processed in 24us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.790756:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2449 00000100:00100000:1.0:1769074853.790765:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2450 00000100:00100000:1.0:1769074853.790771:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625239424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790799:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625239424:12345-0@lo:400:kworker.0 Request processed in 28us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.790830:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2450 00000100:00080000:1.0:1769074853.790860:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074853.790865:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074853.790895:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625238912:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790905:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625239424:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790914:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625239296:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.790922:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625239808:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791110:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625238656:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791148:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074853.791187:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625239168:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791211:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074853.791229:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625239680:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791248:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074853.791264:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1769074853.791270:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625239040:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791291:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074853.791306:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625239552:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791326:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074853.791350:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239168 00000100:00100000:3.0:1769074853.791354:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239168 00000100:00100000:3.0:1769074853.791382:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239040 00000100:00100000:3.0:1769074853.791385:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239040 00000100:00100000:3.0:1769074853.791404:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1541 00000100:00100000:3.0:1769074853.791410:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625239168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791445:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625239168:12345-0@lo:400:kworker.0 Request processed in 35us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074853.791451:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1541 00000100:00100000:3.0:1769074853.791477:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625239168:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791493:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1542 00000100:00100000:3.0:1769074853.791499:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625239040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.791529:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625239040:12345-0@lo:400:kworker.0 Request processed in 30us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074853.791534:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1542 00000100:00100000:3.0:1769074853.791553:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625239040:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.791892:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625238656 00000100:00100000:1.0:1769074853.791897:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625238656 00000100:00100000:1.0:1769074853.791933:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:1.0:1769074853.791939:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f81850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625238656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.791991:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f81850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625238656:12345-0@lo:400:kworker.0 Request processed in 52us (841us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.791998:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1769074853.792018:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239680 00000100:00100000:2.0:1769074853.792029:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239680 00000100:00100000:1.0:1769074853.792037:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625239552 00000100:00100000:1.0:1769074853.792043:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625239552 00000100:00100000:2.0:1769074853.792071:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2451 00000100:00100000:1.0:1769074853.792078:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2452 00000100:00100000:2.0:1769074853.792083:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625239680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.792091:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625239552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074853.792136:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625239680:12345-0@lo:400:kworker.0 Request processed in 56us (885us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074853.792141:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625239552:12345-0@lo:400:kworker.0 Request processed in 52us (811us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074853.792149:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2451 00000100:00080000:3.0:1769074853.792209:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074853.792213:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074853.792215:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074853.792217:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074853.792238:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625239552:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.792246:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625238656:0@lo:400:kworker.0 00000100:00100000:3.0:1769074853.792254:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625239680:0@lo:400:kworker.0 00000100:00100000:0.0:1769074853.792348:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625238784 00000100:00100000:0.0:1769074853.792353:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625238784 00000100:00100000:0.0:1769074853.792392:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2723 00000100:00100000:0.0:1769074853.792399:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625238784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074853.792458:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625238784:12345-0@lo:400:kworker.0 Request processed in 59us (2231us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074853.792463:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2723 00000100:00100000:3.0:1769074853.792531:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625238784:0@lo:400:kworker.0 00000100:00100000:1.0:1769074853.799533:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2452 00000100:00100000:1.0:1769074853.830794:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074853.830875:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617032960 00000100:00100000:0.0:1769074853.830887:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617032960 00002000:00100000:0.0:1769074853.830939:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x2c6:0x0] (1048576->2097151) req@ffff89b545389880 x1855007617032960/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:389/0 lens 488/0 e 0 to 0 dl 1769074864 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074853.830961:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545389880 with x1855007617032960 ext(1048576->2097151) 00002000:00010000:0.0:1769074853.830967:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545389880 00000100:00100000:0.0:1769074853.831001:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8756 00000100:00100000:0.0:1769074853.831010:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617032960:12345-192.168.204.60@tcp:4:dd.0 00000100:00100000:0.0:1769074853.870308:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617032960:12345-192.168.204.60@tcp:4:dd.0 Request processed in 39278us (39503us total) trans 17179874633 rc 0/0 00000100:00100000:0.0:1769074853.870322:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8756 00002000:00100000:0.0:1769074853.870335:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x2c6:0x0] (1048576->2097151) req@ffff89b545389880 x1855007617032960/t17179874633(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:389/0 lens 488/448 e 0 to 0 dl 1769074864 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1769074853.870346:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b545389880 with x1855007617032960 ext(1048576->2097151) 00002000:00010000:0.0:1769074853.870352:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b545389880 00010000:00010000:1.0:1769074854.302784:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1769074854.310350:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968369 is committed 00000100:00100000:1.0:1769074854.852176:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074854.852219:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074854.852254:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074854.852272:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074854.852305:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074854.852334:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033600 00000100:00100000:1.0:1769074854.852344:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033600 00000100:00100000:1.0:1769074854.852386:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033472 00000100:00100000:1.0:1769074854.852389:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033472 00000100:00100000:1.0:1769074854.852408:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1543 00000100:00100000:1.0:1769074854.852416:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617033600:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074854.852490:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617033600:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 74us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074854.852498:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1543 00000100:00100000:1.0:1769074854.852516:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1544 00000100:00100000:1.0:1769074854.852522:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617033472:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074854.852654:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033344 00000100:00100000:0.0:1769074854.852659:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033344 00000100:00100000:0.0:1769074854.852694:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033216 00000100:00100000:0.0:1769074854.852697:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033216 00000100:00100000:0.0:1769074854.852717:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2453 00000100:00100000:0.0:1769074854.852723:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007617033344:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074854.852755:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:2409:x1855007617033344:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074854.852762:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2453 00000100:00100000:0.0:1769074854.852776:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2454 00000100:00100000:0.0:1769074854.852782:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007617033216:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074854.852856:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:2409:x1855007617033216:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 73us (582us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074854.852862:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2454 00000100:00100000:3.0:1769074854.853183:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033088 00000100:00100000:3.0:1769074854.853189:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033088 00000100:00100000:3.0:1769074854.853231:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 375 00000100:00100000:3.0:1769074854.853239:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f82850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007617033088:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074854.853275:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f82850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007617033088:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 37us (1108us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074854.853281:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 375 00000100:00080000:3.0:1769074854.853315:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074854.853948:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617033472:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 1422us (1638us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074854.853958:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1544 00010000:00010000:2.0:1769074855.325088:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1769074855.486077:0:37997:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 204c: FLR write/stale/resync test with component removal ========================================================== 04:40:53 (1769074853) 00000100:00100000:0.0:1769074855.714536:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074855.715961:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033728 00000100:00100000:3.0:1769074855.715975:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033728 00000100:00100000:3.0:1769074855.716061:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13590 00000100:00100000:3.0:1769074855.716072:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:67391:x1855007617033728:12345-192.168.204.60@tcp:41:stat.0 00000004:00000024:3.0:1769074855.716125:0:8855:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7827804160 avail 7363686400 00000100:00100000:3.0:1769074855.716204:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:67391:x1855007617033728:12345-192.168.204.60@tcp:41:stat.0 Request processed in 134us (1676us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074855.716212:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13590 00000100:00080000:0.0:1769074855.716278:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074855.802570:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074855.802663:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617033856 00000100:00100000:0.0:1769074855.802674:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617033856 00000100:00100000:0.0:1769074855.802750:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13591 00000100:00100000:0.0:1769074855.802760:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:67395:x1855007617033856:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074855.802779:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074855.802798:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd558ed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c878 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.802905:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.802933:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.802945:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.802963:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.802969:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.802975:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803018:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803025:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803032:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803039:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803044:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074855.803049:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803057:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803067:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074855.803071:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074855.803076:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803081:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd558f4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803095:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803101:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803107:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803112:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074855.803116:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803122:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803128:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4fb8ef17 (ffff89b67a317500) 00010000:00010000:0.0:1769074855.803131:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074855.803133:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803139:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd558fb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.803179:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd558ed lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c878 expref: 18 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.803188:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074855.803190:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074855.803194:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd558ed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c878 expref: 18 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.803207:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134e00, rc 0) 00000100:00100000:0.0:1769074855.803266:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:67395:x1855007617033856:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 506us (705us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074855.803273:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13591 00000100:00100000:1.0:1769074855.813289:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074855.813409:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034112 00000100:00100000:2.0:1769074855.813420:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034112 00000100:00100000:2.0:1769074855.813580:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13592 00000100:00100000:2.0:1769074855.813591:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+17:67395:x1855007617034112:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074855.813646:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074855.813673:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd5549b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c759 expref: 17 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.813692:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074855.813696:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074855.813771:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.813784:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.813794:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.813846:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.813853:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.813860:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000040:00080000:2.0:1769074855.817575:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].86, 2168 off92808 00000040:00080000:2.0:1769074855.817670:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000405:0x2:0x0].8, 2368 off49728 00000004:00080000:2.0:1769074855.817679:0:6513:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b5410b83c0 with version 21 00000004:00080000:3.0:1769074855.818046:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 21 rpc_version 21 00000020:00100000:3.0:1769074855.818083:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1769074855.818094:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b5461e7b80 x1855007625240064/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1769074855.818117:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e7b80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625240064:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1769074855.818206:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074855.818236:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7e240 going to sleep for 11 seconds 00000100:00100000:2.0:1769074855.818447:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240064 00000100:00100000:2.0:1769074855.818455:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240064 00000100:00100000:2.0:1769074855.818510:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2724 00000100:00100000:2.0:1769074855.818519:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625240064:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1769074855.819358:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625240064:12345-0@lo:1000:osp_up1-0.0 Request processed in 837us (1150us total) trans 4294968370 rc 0/0 00000100:00100000:2.0:1769074855.819367:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2724 00000100:00100000:3.0:1769074855.819781:0:32392:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968369 gen 2 00000004:00080000:3.0:1769074855.819787:0:32392:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968369 committed transno 4294968369 00000100:00100000:3.0:1769074855.819842:0:32392:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b5461e7b80 x1855007625240064/t4294968370(4294968370) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 1769074871 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769074855.819888:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e7b80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625240064:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1769074855.820841:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x209:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820858:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820871:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820889:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820896:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820903:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820946:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 3/0,1 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820958:0:6513:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 3/0,1 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.820967:0:6513:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 3/0,1 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6513 initiator: MDT0 00000100:00100000:2.0:1769074855.820986:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6513:1855007625240192:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1769074855.821033:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074855.821071:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769074855.821965:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240192 00000100:00100000:3.0:1769074855.821975:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240192 00000100:00100000:3.0:1769074855.822033:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2725 00000100:00100000:3.0:1769074855.822042:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6513:x1855007625240192:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1769074855.822065:0:6517:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074855.822111:0:6517:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 2/0,0 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7fe2323adfd5591e expref: -99 pid: 6517 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.822133:0:6517:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7fe2323adfd5591e expref: 7 pid: 6517 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.822145:0:6517:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd5591e expref: 7 pid: 6517 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.822154:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x9:0x0].0x0 (ffff89b6543eaa00) 00010000:00010000:3.0:1769074855.822157:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074855.822171:0:6517:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29a00, rc 0) 00000100:00100000:3.0:1769074855.822224:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6513:x1855007625240192:12345-0@lo:101:mdt00_001.0 Request processed in 184us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074855.822232:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2725 00000100:00100000:2.0:1769074855.822543:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6513:1855007625240192:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1769074855.822559:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.822568:0:6513:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1769074855.822592:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000096ac4532 reply state 00000000c6c40999 transno 8589944118 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822604:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822610:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074855.822613:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074855.822616:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074855.822622:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 4/0,1 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.822631:0:6513:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1769074855.822645:0:6513:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000405:0x9:0x0] 00000004:00010000:2.0:1769074855.822652:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000096ac4532 reply state 00000000c6c40999 transno 8589944118 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822660:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822666:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x209:0x0].0x0 (ffff89b64effa500) 00010000:00010000:2.0:1769074855.822668:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074855.822670:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1769074855.822675:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x209:0x0]: returning size 4096 00010000:00010000:2.0:1769074855.822685:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822691:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822696:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.822703:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.824610:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074855.824618:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.824627:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.824645:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074855.824647:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074855.824653:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.824660:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55902 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000004:00010000:2.0:1769074855.824685:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000096ac4532 reply state 00000000c6c40999 transno 8589944118 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.824697:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.824704:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4fb8ef17 (ffff89b64effa700) 00010000:00010000:2.0:1769074855.824718:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074855.824722:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1769074855.825031:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:67395:x1855007617034112:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 11439us (11749us total) trans 8589944118 rc 0/0 00000100:00100000:2.0:1769074855.825040:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13592 00000100:00100000:3.0:1769074855.837290:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625240320:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074855.837365:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074855.837766:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240320 00000100:00100000:2.0:1769074855.837775:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240320 00000100:00100000:2.0:1769074855.837855:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2726 00000100:00100000:2.0:1769074855.837866:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625240320:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074855.837899:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074855.837962:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625240320:12345-0@lo:41:osp-pre-1-0.0 Request processed in 98us (596us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074855.837969:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2726 00000100:00100000:1.0:1769074855.838108:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625240320:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074855.838281:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074855.838293:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625240448:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074855.838321:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074855.838565:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240448 00000100:00100000:0.0:1769074855.838571:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240448 00000100:00100000:0.0:1769074855.838609:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2727 00000100:00100000:0.0:1769074855.838618:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625240448:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074855.838637:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:0.0:1769074855.838684:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625240448:12345-0@lo:41:osp-pre-0-1.0 Request processed in 67us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074855.838690:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2727 00000100:00100000:3.0:1769074855.838763:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625240448:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074855.876101:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074855.876239:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034240 00000100:00100000:2.0:1769074855.876249:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034240 00000100:00100000:2.0:1769074855.876347:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13593 00000100:00100000:2.0:1769074855.876357:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:67398:x1855007617034240:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074855.876382:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074855.876402:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5592c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c87f expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.876481:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55933 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.876495:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55933 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.876506:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55933 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074855.876557:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074855.876561:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074855.876578:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55933 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.876599:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5592c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c87f expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.876615:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55933 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c87f expref: 19 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.876624:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074855.876626:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074855.876630:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074855.876683:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:67398:x1855007617034240:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 328us (592us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074855.876689:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13593 00000100:00100000:3.0:1769074855.880542:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074855.880722:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034368 00000100:00100000:2.0:1769074855.880733:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034368 00000100:00100000:2.0:1769074855.880800:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13594 00000100:00100000:2.0:1769074855.880848:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:67398:x1855007617034368:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074855.880868:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074855.880889:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5593a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c886 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.880971:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.880984:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.880995:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881005:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881010:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881015:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881097:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5594f lrc: 3/1,0 mode: --/PR res: [0x200000405:0x209:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881105:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5594f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881111:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5594f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074855.881198:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x209:0x0]: returning size 4096 00000004:00200000:2.0:1769074855.881203:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1769074855.881221:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5594f lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881232:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881238:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881243:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881249:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881254:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074855.881258:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881264:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881275:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074855.881278:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074855.881283:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881289:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55941 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881303:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881309:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881314:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881318:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074855.881320:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881325:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881331:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4fb8ef17 (ffff89b64effa700) 00010000:00010000:2.0:1769074855.881334:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074855.881336:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881342:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55948 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.881366:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd5593a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c886 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.881379:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88400/0x7fe2323adfd5594f lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c886 expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.881386:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x209:0x0].0x0 (ffff89b64effa500) 00010000:00010000:2.0:1769074855.881388:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1769074855.881390:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88400, rc 0) 00000100:00100000:2.0:1769074855.881446:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:67398:x1855007617034368:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 600us (914us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074855.881454:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13594 00000100:00100000:1.0:1769074855.886713:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074855.890337:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034496 00000100:00100000:0.0:1769074855.890423:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034496 00000100:00100000:0.0:1769074855.890544:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13595 00000100:00100000:0.0:1769074855.890555:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:67398:x1855007617034496:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074855.890575:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074855.890597:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55956 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c88d expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.890656:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5595d lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.890669:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5595d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.890680:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5595d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:0.0:1769074855.890731:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd0:0x9:0x0]: returning size 4096 00000004:00200000:0.0:1769074855.890736:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1769074855.890769:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55956 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c88d expref: 22 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.890788:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5595d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c88d expref: 21 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.890797:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074855.890800:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1769074855.890821:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134c00, rc 0) 00000100:00100000:0.0:1769074855.890959:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67398:x1855007617034496:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 406us (4266us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074855.890967:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13595 00000100:00100000:1.0:1769074855.895623:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074855.895707:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034624 00000100:00100000:0.0:1769074855.895717:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034624 00000100:00100000:0.0:1769074855.895785:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13596 00000100:00100000:0.0:1769074855.895795:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+6:67398:x1855007617034624:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074855.895842:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074855.895864:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55964 lrc: 2/0,0 mode: --/CR res: [0x240000405:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c894 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.895936:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/1,0 mode: --/PR res: [0x240000405:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.895956:0:8855:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 2/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd5591e expref: 6 pid: 6517 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.895969:0:8855:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 4/1,0 mode: --/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT1 00010000:00010000:0.0:1769074855.896002:0:8855:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7fe2323adfd5591e expref: 6 pid: 6517 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074855.896021:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:8855:1855007625240576:0@lo:104: 00000100:00100000:0.0:1769074855.896068:0:8855:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074855.896317:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b643015540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074855.896693:0:6496:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240576 00000100:00100000:3.0:1769074855.896700:0:6496:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240576 00000100:00100000:3.0:1769074855.896884:0:6496:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:3.0:1769074855.896893:0:6496:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:8855:x1855007625240576:12345-0@lo:104: 00010000:00010000:3.0:1769074855.896909:0:6496:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.897137:0:6496:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1769074855.897143:0:6496:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:8855:x1855007625240576:12345-0@lo:104: Request processed in 252us (1075us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074855.897151:0:6496:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00010000:00010000:2.0:1769074855.897202:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074855.897209:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 3/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.897217:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc1908290) 00010000:00010000:2.0:1769074855.897223:0:8413:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1769074855.897231:0:8413:0:(mdt_handler.c:3944:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.897259:0:8413:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 2/0,0 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.897265:0:8413:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1769074855.897285:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 1/0,0 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.897292:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd5591e lrc: 0/0,0 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd55925 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.897310:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074855.897311:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074855.897313:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074855.897321:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074855.897331:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625240704:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1769074855.897366:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074855.897398:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240704 00000100:00100000:2.0:1769074855.897402:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240704 00010000:00010000:2.0:1769074855.897423:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7fe2323adfd5591e expref: 7 pid: 6517 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074855.897446:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7222 00000100:00100000:2.0:1769074855.897451:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625240704:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1769074855.897462:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1769074855.897943:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:8855:1855007625240576:0@lo:104: 00010000:00010000:0.0:1769074855.897964:0:8855:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/1,0 mode: --/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT1 00000001:00080000:1.0:1769074855.902580:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968370 is committed 00010000:00010000:2.0:1769074855.902717:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b543a29a00/0x7fe2323adfd55925 lrc: 0/0,0 mode: --/PW res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7fe2323adfd5591e expref: 6 pid: 6517 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.902736:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x9:0x0].0x0 (ffff89b6543eaa00) 00010000:00010000:2.0:1769074855.902742:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/1,0 mode: --/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT1 00010000:00010000:2.0:1769074855.902751:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT1 00010000:00010000:2.0:1769074855.902759:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 4/1,0 mode: PR/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8855 initiator: MDT1 00010000:00010000:2.0:1769074855.902797:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074855.902860:0:8855:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT1 00010000:00010000:0.0:1769074855.902870:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT1 00000100:00100000:2.0:1769074855.902942:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625240704:12345-0@lo:103:ldlm_bl.0 Request processed in 5488us (5573us total) trans 0 rc 0/0 00000004:00200000:0.0:1769074855.902948:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000405:0x9:0x0]: returning size 4096 00000100:00100000:2.0:1769074855.902954:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7222 00000004:00200000:0.0:1769074855.902955:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1769074855.903005:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55964 lrc: 0/0,0 mode: --/CR res: [0x240000405:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c894 expref: 8 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.903022:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135600/0x7fe2323adfd5596b lrc: 3/0,0 mode: PR/PR res: [0x240000405:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c894 expref: 7 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.903030:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0x9:0x0].0x0 (ffff89b6543eaa00) 00010000:00010000:0.0:1769074855.903033:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1769074855.903036:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135600, rc 0) 00000100:00100000:1.0:1769074855.903086:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625240704:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1769074855.903097:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:67398:x1855007617034624:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 7302us (7481us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074855.903109:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13596 00000100:00100000:3.0:1769074855.905340:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074855.905496:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034752 00000100:00100000:0.0:1769074855.905506:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034752 00000100:00100000:0.0:1769074855.905561:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13597 00000100:00100000:0.0:1769074855.905571:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67398:x1855007617034752:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074855.905591:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074855.905612:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55972 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c89b expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.905678:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905693:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905704:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905752:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905760:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905766:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905861:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905873:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905879:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905885:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905890:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074855.905894:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905901:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905922:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074855.905925:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074855.905930:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905936:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55979 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905949:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905954:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905959:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905963:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074855.905966:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905971:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905977:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b67a317500) 00010000:00010000:0.0:1769074855.905980:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074855.905982:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.905988:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55980 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074855.906016:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55972 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c89b expref: 22 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.906026:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:0.0:1769074855.906028:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074855.906032:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134e00/0x7fe2323adfd55972 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c89b expref: 22 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074855.906041:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549134e00, rc 0) 00000100:00100000:0.0:1769074855.906096:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67398:x1855007617034752:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 526us (764us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074855.906104:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13597 00000100:00100000:2.0:1769074855.912004:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074855.912246:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617034880 00000100:00100000:3.0:1769074855.912473:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617034880 00000100:00100000:3.0:1769074855.912533:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13598 00000100:00100000:3.0:1769074855.912543:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67398:x1855007617034880:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074855.912563:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074855.912583:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55987 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8a2 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.912650:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5598e lrc: 3/1,0 mode: --/PR res: [0x200000405:0x209:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.912665:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5598e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.912676:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5598e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074855.912781:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x209:0x0]: returning size 4096 00000004:00200000:3.0:1769074855.912787:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1769074855.912828:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5598e lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.912854:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55987 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a2 expref: 23 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.912870:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28000/0x7fe2323adfd5598e lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x209:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a2 expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.912879:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x209:0x0].0x0 (ffff89b64effa500) 00010000:00010000:3.0:1769074855.912882:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074855.912886:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28000, rc 0) 00000100:00100000:3.0:1769074855.913190:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67398:x1855007617034880:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 647us (1195us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074855.913198:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13598 00000100:00100000:1.0:1769074855.919113:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074855.919542:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617035136 00000100:00100000:3.0:1769074855.919553:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617035136 00000100:00100000:3.0:1769074855.919650:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13599 00000100:00100000:3.0:1769074855.919660:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67398:x1855007617035136:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074855.919681:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074855.919685:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074855.919712:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134c00/0x7fe2323adfd5595d lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c88d expref: 22 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.919741:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074855.919746:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074855.919760:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55995 lrc: 2/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8a9 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074855.920083:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 3/0,1 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.920098:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.920108:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.920123:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.920131:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.920138:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921027:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921042:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921051:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:3.0:1769074855.921061:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 0 00010000:00010000:3.0:1769074855.921087:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921094:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921100:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 3/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921107:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921113:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a29800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074855.921118:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921125:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 5/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921136:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:3.0:1769074855.921139:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074855.921143:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 3/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921149:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd5599c lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000004:00010000:3.0:1769074855.921431:0:6513:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008dbb7ba3 reply state 0000000002311686 transno 8589944119 ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921623:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074855.921630:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b6543eac00) 00010000:00010000:3.0:1769074855.921633:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074855.921637:0:6513:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1769074855.922005:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29400/0x7fe2323adfd55995 lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 24 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074855.922025:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 23 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074855.922033:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:1.0:1769074855.922036:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074855.922038:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28400, rc 0) 00000100:00100000:1.0:1769074855.922174:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67398:x1855007617035136:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 2514us (3071us total) trans 8589944119 rc 0/0 00000100:00100000:1.0:1769074855.922183:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13599 00000100:00100000:1.0:1769074855.927075:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074855.927427:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617035392 00000100:00100000:2.0:1769074855.927505:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617035392 00000100:00100000:2.0:1769074855.927614:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13600 00000100:00100000:2.0:1769074855.927624:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67398:x1855007617035392:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074855.927648:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074855.927669:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd559b8 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8b0 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.927789:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.927904:0:6513:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.927923:0:6513:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 4/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.927955:0:6513:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.927969:0:6513:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 25 pid: 6513 timeout: 1943 lvb_type: 0 00000100:00100000:2.0:1769074855.927982:0:6513:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625240832:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074855.928096:0:6513:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb7f180 going to sleep for 11 seconds 00000100:00100000:2.0:1769074855.934865:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074855.935654:0:6513:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6513:1855007625240832:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074855.935685:0:6513:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000100:00100000:0.0:1769074855.937168:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617035520 00000100:00100000:0.0:1769074855.937181:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617035520 00010000:00010000:0.0:1769074855.937229:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 26 pid: 6513 timeout: 1943 lvb_type: 0 00000100:00100000:0.0:1769074855.937298:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7223 00000100:00100000:0.0:1769074855.937308:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:2409:x1855007617035520:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074855.937325:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074855.937331:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 26 pid: 6513 timeout: 1943 lvb_type: 0 00010000:00010000:0.0:1769074855.937344:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 26 pid: 6513 timeout: 1943 lvb_type: 0 00010000:00010000:0.0:1769074855.937365:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28400/0x7fe2323adfd559aa lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8a9 expref: 26 pid: 6513 timeout: 1943 lvb_type: 0 00010000:00010000:0.0:1769074855.937381:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:0.0:1769074855.937387:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074855.937396:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074855.937403:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074855.937441:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074855.937502:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:2409:x1855007617035520:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 195us (2647us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074855.937510:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7223 00010000:00010000:2.0:1769074855.938299:0:6513:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938311:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:2.0:1769074855.938333:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 0 00000004:00080000:2.0:1769074855.938500:0:6513:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2c9:0x0], index 0: 0 00000004:00080000:2.0:1769074855.938508:0:6513:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2cb:0x0], index 1: 0 00010000:00010000:2.0:1769074855.938614:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938623:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938631:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938637:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074855.938641:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938648:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938660:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:2.0:1769074855.938663:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074855.938667:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 2/0,0 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938672:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd559bf lrc: 0/0,0 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074855.938744:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd559b8 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8b0 expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.938753:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:2.0:1769074855.938755:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074855.938758:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd559b8 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8b0 expref: 25 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074855.938772:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88c00, rc 0) 00000100:00100000:2.0:1769074855.938884:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54538bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67398:x1855007617035392:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11261us (11819us total) trans 8589944120 rc 301/301 00000100:00100000:2.0:1769074855.938893:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13600 00000100:00100000:2.0:1769074855.955519:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074855.955612:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617035776 00000100:00100000:0.0:1769074855.955623:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617035776 00000100:00100000:0.0:1769074855.955719:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4742 00000100:00100000:0.0:1769074855.955738:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67398:x1855007617035776:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074855.956208:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67398:x1855007617035776:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 478us (697us total) trans 8589944121 rc 0/0 00000100:00100000:0.0:1769074855.956216:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4742 00000100:00100000:1.0:1769074855.959171:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074855.959612:0:6515:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036032 00000100:00100000:2.0:1769074855.959623:0:6515:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036032 00000100:00100000:2.0:1769074855.959676:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4743 00000100:00100000:2.0:1769074855.959687:0:6515:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67398:x1855007617036032:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074855.959982:0:6515:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:67398:x1855007617036032:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 296us (831us total) trans 8589944122 rc 0/0 00000100:00100000:2.0:1769074855.959990:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4743 00010000:00010000:0.0:1769074856.351015:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1769074856.351283:0:32395:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1844193474889ns, next at 1845193472269ns, rc = 0 00000100:00100000:0.0:1769074856.351340:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625240960:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074856.351422:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074856.351474:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074856.351481:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625241088:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074856.351511:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074856.351546:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625240960 00000100:00100000:0.0:1769074856.351553:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625240960 00000100:00100000:0.0:1769074856.351604:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625241088 00000100:00100000:0.0:1769074856.351607:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625241088 00000100:00100000:0.0:1769074856.351639:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1232 00000100:00100000:0.0:1769074856.351647:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625240960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074856.351675:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3913945088 avail 3682422784 00000100:00100000:0.0:1769074856.351730:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625240960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074856.351738:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1232 00000100:00100000:0.0:1769074856.351755:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1233 00000100:00100000:0.0:1769074856.351762:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625241088:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074856.351772:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3913945088 avail 3682422784 00000100:00100000:0.0:1769074856.351796:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625241088:12345-0@lo:13:osp-pre-0-1.0 Request processed in 35us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074856.351804:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1233 00000100:00080000:0.0:1769074856.351847:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074856.351887:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625241088:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074856.351902:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625240960:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074856.377394:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074856.377420:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625241216:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074856.377494:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074856.377545:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625241216 00000100:00100000:0.0:1769074856.377555:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625241216 00000100:00100000:0.0:1769074856.377610:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1234 00000100:00100000:0.0:1769074856.377620:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625241216:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074856.377650:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 38180160 pending 1073152 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074856.377709:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625241216:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074856.377718:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1234 00000100:00100000:0.0:1769074856.377763:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874632 gen 2 00000004:00080000:0.0:1769074856.377769:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a271c00, transno 17179874630 00000004:00080000:0.0:1769074856.377777:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a273800, transno 17179874631 00000004:00080000:0.0:1769074856.377780:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a270e00, transno 17179874632 00000100:00100000:0.0:1769074856.377791:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625241216:0@lo:13:osp-pre-1-0.0 00000004:00080000:0.0:1769074856.377935:0:32396:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 3 done 00000100:00080000:0.0:1769074856.377952:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:0.0:1769074856.377963:0:32397:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1844220156525ns, next at 1845220154830ns, rc = 0 00000100:00100000:1.0:1769074856.864374:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625241344:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074856.864441:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074856.864552:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625241344 00000100:00100000:0.0:1769074856.864560:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625241344 00000100:00100000:0.0:1769074856.864629:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1235 00000100:00100000:0.0:1769074856.864638:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625241344:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074856.864659:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 38180160 pending 1073152 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074856.864705:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625241344:12345-0@lo:13:osp-pre-1-1.0 Request processed in 69us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074856.865112:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1235 00000100:00080000:1.0:1769074856.865264:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074856.865303:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625241344:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074857.373008:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074858.399607:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1769074858.912648:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1970 committed 00000004:00080000:3.0:1769074858.912682:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1968 committed 00000001:00080000:3.0:1769074858.912706:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944122 is committed 00000100:00100000:3.0:1769074858.912841:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625241472:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1769074858.912916:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074858.913449:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625241472 00000100:00100000:1.0:1769074858.913456:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625241472 00000100:00100000:1.0:1769074858.913509:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14520 00000100:00100000:1.0:1769074858.913520:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625241472:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1769074858.913544:0:9048:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:712 count 1 00010000:00010000:1.0:1769074858.913572:0:9048:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.913592:0:9048:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.913600:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074858.913603:0:9048:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074858.913606:0:9048:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.913633:0:9048:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.913646:0:9048:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 8 pid: 36516 timeout: 1946 lvb_type: 0 00000100:00100000:1.0:1769074858.913658:0:9048:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625241728:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074858.913724:0:9048:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b492d80 going to sleep for 11 seconds 00000100:00080000:1.0:1769074858.913737:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074858.913916:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625241600:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1769074858.913952:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074858.914373:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625241600 00000100:00100000:0.0:1769074858.914380:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625241600 00000100:00100000:0.0:1769074858.914520:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14521 00000100:00100000:0.0:1769074858.914530:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625241600:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1769074858.914548:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:714 count 1 00010000:00010000:0.0:1769074858.914568:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074858.914584:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074858.914591:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074858.914593:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074858.914595:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074858.914618:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 9 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074858.914629:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 9 pid: 27843 timeout: 1946 lvb_type: 0 00000100:00100000:0.0:1769074858.914647:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625241856:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074858.914705:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b66e69bc00 going to sleep for 11 seconds 00010000:00010000:2.0:1769074858.917842:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917874:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917882:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917888:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074858.917893:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917900:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917916:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4fb8ef17 (ffff89b64effa700) 00010000:00010000:2.0:1769074858.917920:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074858.917925:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917931:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55909 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x4fb8ef17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917958:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917964:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917969:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917974:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074858.917977:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917982:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.917989:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x209:0x0].0x0 (ffff89b64effa500) 00010000:00010000:2.0:1769074858.917992:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074858.917995:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918000:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55910 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x209:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918011:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918029:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918040:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918045:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074858.918047:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918052:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918059:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074858.918061:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074858.918064:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918069:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55917 lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918087:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918094:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918100:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918105:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074858.918107:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918113:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918126:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b6543eac00) 00010000:00010000:2.0:1769074858.918129:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074858.918132:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:2.0:1769074858.918138:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28a00/0x7fe2323adfd559a3 lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00000100:00100000:2.0:1769074858.919113:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625242368:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.919571:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074858.919611:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625242880:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.919639:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074858.919667:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1769074858.919673:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625242240:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.919696:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074858.919712:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625242752:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.919732:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074858.919752:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625243264:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.919772:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074858.919825:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242880 00000100:00100000:2.0:1769074858.919834:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242880 00000100:00100000:0.0:1769074858.919962:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625241984:0@lo:400:kworker.0 00000100:00100000:0.0:1769074858.919996:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074858.920041:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625242496:0@lo:400:kworker.0 00000100:00100000:1.0:1769074858.920050:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625242112:0@lo:400:kworker.0 00000100:00100000:0.0:1769074858.920067:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074858.920090:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625243008:0@lo:400:kworker.0 00000100:00100000:1.0:1769074858.920106:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074858.920119:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074858.920138:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625242624:0@lo:400:kworker.0 00000100:00100000:0.0:1769074858.920156:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242240 00000100:00100000:1.0:1769074858.920165:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074858.920172:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242240 00000100:00100000:1.0:1769074858.920180:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625243136:0@lo:400:kworker.0 00000100:00100000:0.0:1769074858.920209:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242752 00000100:00100000:0.0:1769074858.920214:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242752 00000100:00100000:1.0:1769074858.920215:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074858.920249:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242368 00000100:00100000:1.0:1769074858.920254:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242368 00000100:00080000:3.0:1769074858.927063:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074858.927074:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074858.927078:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074858.927082:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074858.927158:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243264 00000100:00100000:3.0:1769074858.927645:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243264 00000100:00100000:3.0:1769074858.927684:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243008 00000100:00100000:3.0:1769074858.927687:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243008 00000100:00100000:3.0:1769074858.927710:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242624 00000100:00100000:3.0:1769074858.927714:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242624 00000100:00100000:3.0:1769074858.927734:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2455 00000100:00100000:3.0:1769074858.927743:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625242240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.927859:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625242240:12345-0@lo:400:kworker.0 Request processed in 118us (8161us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.927878:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2455 00000100:00100000:3.0:1769074858.927896:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2456 00000100:00100000:3.0:1769074858.927902:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625243264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.927933:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625243264:12345-0@lo:400:kworker.0 Request processed in 32us (8160us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.927939:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2456 00000100:00100000:3.0:1769074858.927951:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2457 00000100:00100000:3.0:1769074858.927957:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625243008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.927996:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625243008:12345-0@lo:400:kworker.0 Request processed in 39us (7877us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.928002:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2457 00000100:00100000:3.0:1769074858.928898:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2458 00000100:00100000:3.0:1769074858.928912:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625242624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.928966:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625242624:12345-0@lo:400:kworker.0 Request processed in 59us (8800us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.928975:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2458 00000100:00080000:3.0:1769074858.929001:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074858.929005:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074858.929056:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625242624:0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929117:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625242240:0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929129:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625243264:0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929407:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625243008:0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929430:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242112 00000100:00100000:3.0:1769074858.929434:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242112 00000100:00100000:3.0:1769074858.929614:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243136 00000100:00100000:3.0:1769074858.929619:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243136 00000100:00100000:3.0:1769074858.929638:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2728 00000100:00100000:3.0:1769074858.929643:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625242112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929687:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625242112:12345-0@lo:400:kworker.0 Request processed in 43us (9593us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.929693:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2728 00000100:00100000:3.0:1769074858.929707:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2729 00000100:00100000:3.0:1769074858.929713:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625243136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929741:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625243136:12345-0@lo:400:kworker.0 Request processed in 28us (9534us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.929746:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2729 00000100:00100000:3.0:1769074858.929765:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625242496 00000100:00100000:3.0:1769074858.929768:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625242496 00000100:00100000:3.0:1769074858.929799:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1545 00000100:00100000:3.0:1769074858.929829:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625242496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929879:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625242496:12345-0@lo:400:kworker.0 Request processed in 73us (9811us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.929886:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1545 00000100:00100000:3.0:1769074858.929905:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625241984 00000100:00100000:3.0:1769074858.929908:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625241984 00000100:00100000:3.0:1769074858.929930:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:3.0:1769074858.929935:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f82450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625241984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.929964:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f82450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625241984:12345-0@lo:400:kworker.0 Request processed in 29us (9968us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074858.929970:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:3.0:1769074858.930030:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625241984:0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.930043:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625242496:0@lo:400:kworker.0 00000100:00080000:3.0:1769074858.930061:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074858.930074:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625242112:0@lo:400:kworker.0 00000100:00100000:3.0:1769074858.930083:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968370 gen 2 00000004:00080000:3.0:1769074858.930088:0:3650:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968370 committed transno 4294968370 00000020:00080000:3.0:1769074858.930094:0:3650:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff89b5410b8e40 refcount 2 committed 0 result 0 batchid 4294969368 00000020:00080000:3.0:1769074858.930098:0:3650:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67fd31600 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b5410b83c0 00000020:00080000:3.0:1769074858.930101:0:3650:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000405:0x2:0x0].8 00000020:00080000:3.0:1769074858.930104:0:3650:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b67fd31200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff89b54babe600 00000020:00080000:3.0:1769074858.930107:0:3650:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].86 00000100:00100000:3.0:1769074858.930127:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625243136:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.930897:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2459 00000100:00100000:2.0:1769074858.930909:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625242880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.930960:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625242880:12345-0@lo:400:kworker.0 Request processed in 55us (11320us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074858.930966:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2459 00000100:00100000:2.0:1769074858.931002:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625242880:0@lo:400:kworker.0 00000100:00100000:2.0:1769074858.931181:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074858.931756:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036160 00000100:00100000:2.0:1769074858.931763:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036160 00010000:00010000:2.0:1769074858.931802:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 9 pid: 36516 timeout: 1946 lvb_type: 0 00000100:00100000:2.0:1769074858.931877:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7224 00000100:00100000:2.0:1769074858.931885:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617036160:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074858.931898:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074858.931904:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 9 pid: 36516 timeout: 1946 lvb_type: 0 00010000:00010000:2.0:1769074858.931918:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 9 pid: 36516 timeout: 1946 lvb_type: 0 00010000:00010000:2.0:1769074858.931942:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074858.931945:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074858.931950:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.931960:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.931981:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074858.932023:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617036160:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 139us (844us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074858.932029:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7224 00000100:00080000:2.0:1769074858.932049:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1769074858.932075:0:9048:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55742 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->68157439) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c82b expref: 8 pid: 36516 timeout: 1946 lvb_type: 0 00000100:00100000:2.0:1769074858.932090:0:9048:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9048:1855007625241728:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074858.932107:0:9048:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.932117:0:9048:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.932124:0:9048:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.932133:0:9048:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.932139:0:9048:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.932155:0:9048:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1769074858.934466:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1546 00000100:00100000:1.0:1769074858.934480:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625242368:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074858.934558:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625242368:12345-0@lo:400:kworker.0 Request processed in 81us (15088us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074858.934567:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3651:x1855007625241472:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21046us (21647us total) trans 21474841306 rc 0/0 00000100:00100000:1.0:1769074858.934568:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1546 00000100:00100000:2.0:1769074858.934580:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14520 00000100:00100000:1.0:1769074858.934617:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625242368:0@lo:400:kworker.0 00000004:00080000:2.0:1769074858.934623:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5437b4e00/2, rc 0, transno 21474841306 00000100:00100000:2.0:1769074858.934631:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625241472:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1769074858.934663:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074858.934672:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.934683:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074858.934687:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.934696:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.934714:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.934721:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9048 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074858.934738:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074858.934740:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074858.934742:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1769074858.934854:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294969368 committed batchid 4294969367 00000020:00080000:3.0:1769074858.934915:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969367 00000100:00100000:3.0:1769074858.937800:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625241856:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074858.937855:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074858.940512:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074858.940763:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036288 00000100:00100000:2.0:1769074858.940774:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036288 00010000:00010000:2.0:1769074858.940883:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 10 pid: 27843 timeout: 1946 lvb_type: 0 00000100:00100000:2.0:1769074858.940918:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7225 00000100:00100000:2.0:1769074858.940927:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617036288:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074858.940942:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074858.940948:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 10 pid: 27843 timeout: 1946 lvb_type: 0 00010000:00010000:2.0:1769074858.940961:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 10 pid: 27843 timeout: 1946 lvb_type: 0 00010000:00010000:2.0:1769074858.940984:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549135e00/0x7fe2323adfd5572d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c81d expref: 10 pid: 27843 timeout: 1946 lvb_type: 0 00010000:00010000:2.0:1769074858.941002:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074858.941004:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074858.941008:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941018:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941057:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074858.941117:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617036288:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 190us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074858.941124:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7225 00010000:00010000:2.0:1769074858.941153:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941161:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941171:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941178:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941193:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1769074858.941434:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625241600:12345-0@lo:6:osp-syn-1-0.0 Request processed in 26905us (27480us total) trans 17179874634 rc 0/0 00000100:00100000:2.0:1769074858.941442:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14521 00010000:00010000:2.0:1769074858.941463:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074858.941469:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941476:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074858.941480:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941488:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941499:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941505:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549134600/0x7fe2323adfd559d4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074858.941520:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074858.941521:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074858.941523:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1769074858.942265:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:1.0:1769074858.942309:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54759ca80/2, rc 0, transno 17179874634 00000100:00100000:1.0:1769074858.942317:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625241600:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1769074858.944959:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2460 00000100:00100000:0.0:1769074858.944974:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625242752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074858.945066:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625242752:12345-0@lo:400:kworker.0 Request processed in 95us (25332us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074858.945073:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2460 00000001:00080000:0.0:1769074858.945911:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841305 is committed 00000100:00100000:1.0:1769074858.947179:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625242752:0@lo:400:kworker.0 00000001:00080000:0.0:1769074858.948327:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874634 is committed 00000001:00080000:0.0:1769074858.948337:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874633 is committed 00010000:00010000:2.0:1769074859.424684:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074859.972256:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074859.972300:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074859.972336:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036544 00000100:00100000:2.0:1769074859.972346:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036544 00000100:00100000:3.0:1769074859.972347:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074859.972382:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074859.972405:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036800 00000100:00100000:0.0:1769074859.972410:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036800 00000100:00100000:2.0:1769074859.972416:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036672 00000100:00100000:3.0:1769074859.972419:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074859.972421:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036672 00000100:00100000:0.0:1769074859.972441:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036928 00000100:00100000:0.0:1769074859.972444:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036928 00000100:00100000:2.0:1769074859.972458:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2461 00000100:00100000:3.0:1769074859.972468:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2462 00000100:00100000:0.0:1769074859.972469:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1547 00000100:00100000:2.0:1769074859.972469:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007617036544:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074859.972487:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617036800:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074859.972488:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617036672:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074859.972544:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617036800:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 64us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074859.972552:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1547 00000100:00100000:0.0:1769074859.972566:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1548 00000100:00100000:0.0:1769074859.972571:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617036928:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074859.981431:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2408:x1855007617036544:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 8952us (9171us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074859.981447:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2461 00000100:00100000:2.0:1769074859.981505:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617036416 00000100:00100000:2.0:1769074859.981514:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617036416 00000100:00100000:2.0:1769074859.981557:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 377 00000100:00100000:2.0:1769074859.981567:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b677604850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617036416:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074859.981617:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b677604850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617036416:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 52us (9270us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074859.981624:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 377 00000100:00080000:2.0:1769074859.981799:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074859.981822:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074859.982334:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617036672:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 9850us (9947us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074859.982344:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2462 00000100:00100000:0.0:1769074859.982979:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617036928:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 10402us (10558us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074859.982989:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1548 00010000:00010000:1.0:1769074860.445972:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074860.957299:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074860.957323:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625243392:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074860.957391:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074860.957520:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243392 00000100:00100000:3.0:1769074860.957528:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243392 00000100:00100000:3.0:1769074860.957582:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2730 00000100:00100000:3.0:1769074860.957591:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625243392:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074860.957626:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445363712 avail 1299124224 00000100:00100000:3.0:1769074860.957683:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625243392:12345-0@lo:41:osp-pre-0-1.0 Request processed in 94us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074860.957691:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2730 00000100:00100000:2.0:1769074860.957975:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625243520:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074860.958008:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074860.958079:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243520 00000100:00100000:1.0:1769074860.958084:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243520 00000100:00100000:1.0:1769074860.958120:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2731 00000100:00100000:1.0:1769074860.958127:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625243520:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074860.958145:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:1.0:1769074860.958206:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625243520:12345-0@lo:41:osp-pre-1-0.0 Request processed in 79us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074860.958212:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2731 00000100:00100000:1.0:1769074860.958255:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625243520:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074860.958491:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625243392:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1769074861.469052:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074861.469454:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074861.469477:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625243648:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074861.469550:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074861.469625:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243648 00000100:00100000:0.0:1769074861.469633:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243648 00000100:00100000:0.0:1769074861.469681:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1236 00000100:00100000:0.0:1769074861.469690:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625243648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074861.469722:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074861.469771:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625243648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074861.469778:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1236 00000100:00100000:0.0:1769074861.469845:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874634 gen 2 00000004:00080000:0.0:1769074861.469851:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54759ca80, transno 17179874634 00000100:00100000:0.0:1769074861.469866:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625243648:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1769074861.470402:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074861.470664:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625243904:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074861.470712:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074861.470745:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074861.470751:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625243776:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074861.470773:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074861.470830:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243904 00000100:00100000:3.0:1769074861.470836:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243904 00000100:00100000:3.0:1769074861.470870:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625243776 00000100:00100000:3.0:1769074861.470873:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625243776 00000100:00100000:3.0:1769074861.470894:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1237 00000100:00100000:3.0:1769074861.470901:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625243904:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074861.471106:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074861.471151:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625243904:12345-0@lo:13:osp-pre-0-0.0 Request processed in 249us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074861.471157:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1237 00000100:00100000:3.0:1769074861.471574:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1238 00000100:00100000:3.0:1769074861.471583:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625243776:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074861.471594:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074861.471621:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625243776:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (847us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074861.471626:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1238 00000100:00080000:3.0:1769074861.471645:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074861.471680:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625243776:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074861.471689:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841305 gen 2 00000004:00080000:3.0:1769074861.471693:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5461e6a00, transno 21474841305 00000100:00100000:3.0:1769074861.471716:0:3652:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b5437b4e00 x1855007625241472/t21474841306(21474841306) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1769074874 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769074861.471728:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625243904:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074861.981521:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625244032:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074861.981587:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074861.981993:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244032 00000100:00100000:0.0:1769074861.982003:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244032 00000100:00080000:2.0:1769074861.982106:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074861.982581:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1239 00000100:00100000:0.0:1769074861.982595:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625244032:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074861.982619:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074861.982688:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625244032:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (1100us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074861.982695:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1239 00000100:00100000:2.0:1769074861.982785:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625244032:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074862.495150:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074863.519382:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074864.029402:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074864.030062:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625244416:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.030132:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.030205:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074864.030211:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625244288:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.030238:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.030273:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244288 00000100:00100000:2.0:1769074864.030280:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244288 00000100:00100000:2.0:1769074864.030337:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2732 00000100:00100000:2.0:1769074864.030345:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625244288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.030394:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625244288:12345-0@lo:400:kworker.0 Request processed in 50us (155us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074864.030401:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2732 00000100:00100000:2.0:1769074864.030440:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625244288:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.030516:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244416 00000100:00100000:3.0:1769074864.030521:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244416 00000100:00100000:3.0:1769074864.030556:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2463 00000100:00100000:3.0:1769074864.030563:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625244416:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.030601:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625244416:12345-0@lo:400:kworker.0 Request processed in 38us (469us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.030607:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2463 00000100:00100000:3.0:1769074864.030633:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625244416:0@lo:400:kworker.0 00000100:00100000:0.0:1769074864.031355:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625244544:0@lo:400:kworker.0 00000100:00100000:0.0:1769074864.031394:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074864.031424:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625244160:0@lo:400:kworker.0 00000100:00100000:0.0:1769074864.031448:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.031821:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244544 00000100:00100000:2.0:1769074864.031829:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244544 00000100:00100000:2.0:1769074864.031872:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1549 00000100:00100000:2.0:1769074864.031880:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625244544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.031922:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625244544:12345-0@lo:400:kworker.0 Request processed in 44us (528us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074864.031929:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1549 00000100:00080000:2.0:1769074864.031946:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074864.032086:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625244544:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.033634:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244160 00000100:00100000:3.0:1769074864.033645:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244160 00000100:00100000:3.0:1769074864.033695:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:3.0:1769074864.033705:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b666b8f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625244160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.035312:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b666b8f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625244160:12345-0@lo:400:kworker.0 Request processed in 1601us (3855us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.035336:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:2.0:1769074864.035686:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625244800:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.035737:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.035774:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625245312:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.035827:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.035851:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1769074864.035857:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625244928:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.035877:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.035890:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625245440:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.035910:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074864.035936:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245312 00000100:00100000:2.0:1769074864.035942:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245312 00000100:00100000:2.0:1769074864.035977:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2733 00000100:00100000:2.0:1769074864.035985:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625245312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.036029:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625245312:12345-0@lo:400:kworker.0 Request processed in 45us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074864.036035:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2733 00000100:00100000:2.0:1769074864.036074:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625245312:0@lo:400:kworker.0 00000100:00100000:2.0:1769074864.036097:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245440 00000100:00100000:2.0:1769074864.036100:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245440 00000100:00100000:2.0:1769074864.036132:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1550 00000100:00100000:2.0:1769074864.036138:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625245440:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074864.036674:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244800 00000100:00100000:1.0:1769074864.036680:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244800 00000100:00100000:1.0:1769074864.036713:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244928 00000100:00100000:1.0:1769074864.036716:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244928 00000100:00100000:1.0:1769074864.036737:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2464 00000100:00100000:1.0:1769074864.036745:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625244800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074864.036824:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625244800:12345-0@lo:400:kworker.0 Request processed in 60us (1066us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074864.036832:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2464 00000100:00100000:1.0:1769074864.036847:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2465 00000100:00100000:1.0:1769074864.036853:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625244928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074864.036879:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625244928:12345-0@lo:400:kworker.0 Request processed in 27us (1001us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074864.036886:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2465 00000100:00100000:1.0:1769074864.036917:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625244928:0@lo:400:kworker.0 00000100:00100000:1.0:1769074864.036928:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625244800:0@lo:400:kworker.0 00000100:00080000:1.0:1769074864.036966:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074864.036970:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074864.036972:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074864.038013:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625245440:12345-0@lo:400:kworker.0 Request processed in 1873us (2100us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074864.038021:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1550 00000100:00100000:3.0:1769074864.038960:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625244672:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039012:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074864.039044:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625245184:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039068:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074864.039088:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625245696:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039106:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074864.039128:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1769074864.039134:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625245056:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039255:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074864.039275:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625245568:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039295:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074864.039337:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245184 00000100:00100000:3.0:1769074864.039345:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245184 00000100:00100000:3.0:1769074864.039391:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245696 00000100:00100000:3.0:1769074864.039395:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245696 00000100:00100000:3.0:1769074864.039406:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245056 00000100:00100000:3.0:1769074864.039408:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245056 00000100:00100000:3.0:1769074864.039422:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2466 00000100:00100000:3.0:1769074864.039429:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625245184:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039471:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625245184:12345-0@lo:400:kworker.0 Request processed in 42us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.039478:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2466 00000100:00100000:3.0:1769074864.039493:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2467 00000100:00100000:3.0:1769074864.039498:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625245696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039520:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625245696:12345-0@lo:400:kworker.0 Request processed in 23us (414us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.039525:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2467 00000100:00100000:3.0:1769074864.039532:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2468 00000100:00100000:3.0:1769074864.039537:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625245056:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039557:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1855007625245056:12345-0@lo:400:kworker.0 Request processed in 20us (302us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.039563:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2468 00000100:00080000:3.0:1769074864.039582:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074864.039586:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074864.039620:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3650:1855007625245056:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039630:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625245184:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039639:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625245696:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039665:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625244672 00000100:00100000:3.0:1769074864.039669:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625244672 00000100:00100000:3.0:1769074864.039693:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245568 00000100:00100000:3.0:1769074864.039696:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245568 00000100:00100000:3.0:1769074864.039713:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1551 00000100:00100000:3.0:1769074864.039719:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625244672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039750:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625244672:12345-0@lo:400:kworker.0 Request processed in 31us (738us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.039756:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1551 00000100:00100000:3.0:1769074864.039766:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1552 00000100:00100000:3.0:1769074864.039771:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625245568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039840:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625245568:12345-0@lo:400:kworker.0 Request processed in 69us (543us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074864.039846:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1552 00000100:00080000:3.0:1769074864.039863:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074864.039878:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625245568:0@lo:400:kworker.0 00000100:00100000:3.0:1769074864.039888:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625244672:0@lo:400:kworker.0 00000100:00100000:1.0:1769074864.040002:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625245440:0@lo:400:kworker.0 00000100:00100000:0.0:1769074864.041942:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625244160:0@lo:400:kworker.0 00010000:00010000:2.0:1769074864.541384:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1769074865.056492:0:32377:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 4294969368 batchid updated 00000020:00080000:2.0:1769074865.056545:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969368 00000004:00080000:2.0:1769074865.056607:0:32406:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b6480f3180 with version 22 00000020:00080000:2.0:1769074865.056626:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294969368 cancel update log [0x240000405:0x2:0x0].8: rc = 0 00000004:00080000:1.0:1769074865.056684:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 22 rpc_version 22 00000020:00100000:1.0:1769074865.056742:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1769074865.056772:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b6656d9880 x1855007625245952/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00080000:2.0:1769074865.056802:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294969368 cancel update log [0x200000bd0:0x2:0x0].86: rc = 0 00000100:00100000:1.0:1769074865.056842:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9880 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625245952:0@lo:1000:osp_up1-0.0 00000020:00080000:2.0:1769074865.056862:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969368 00000100:00100000:1.0:1769074865.057220:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074865.057279:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b492480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074865.057416:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625245952 00000100:00100000:2.0:1769074865.057428:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625245952 00000100:00100000:2.0:1769074865.057599:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2734 00000100:00100000:2.0:1769074865.057611:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625245952:12345-0@lo:1000:osp_up1-0.0 00000001:00080000:1.0:1769074865.058000:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841306 is committed 00000100:00100000:2.0:1769074865.058334:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625245952:12345-0@lo:1000:osp_up1-0.0 Request processed in 724us (1205us total) trans 4294968371 rc 0/0 00000100:00100000:2.0:1769074865.058344:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2734 00000100:00100000:1.0:1769074865.058670:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9880 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625245952:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074865.108360:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074865.108406:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074865.108450:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074865.108488:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074865.108514:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074865.108549:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617037184 00000100:00100000:1.0:1769074865.108560:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617037184 00000100:00100000:1.0:1769074865.108622:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617037312 00000100:00100000:1.0:1769074865.108626:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617037312 00000100:00100000:0.0:1769074865.108870:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617037056 00000100:00100000:0.0:1769074865.108875:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617037056 00000100:00100000:0.0:1769074865.108911:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 379 00000100:00100000:0.0:1769074865.108922:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b666b8dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617037056:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074865.109003:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b666b8dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617037056:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 83us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074865.109011:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 379 00000100:00100000:2.0:1769074865.109250:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617037440 00000100:00100000:2.0:1769074865.109255:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617037440 00000100:00100000:2.0:1769074865.109287:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617037568 00000100:00100000:2.0:1769074865.109291:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617037568 00000100:00100000:2.0:1769074865.109314:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1553 00000100:00100000:2.0:1769074865.109321:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617037440:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074865.109411:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2469 00000100:00100000:3.0:1769074865.109420:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007617037184:12345-192.168.204.60@tcp:400:kworker.0 00000100:00080000:0.0:1769074865.111924:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074865.111934:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074865.111938:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074865.111942:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074865.111973:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1554 00000100:00100000:0.0:1769074865.111996:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617037568:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074865.112503:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2470 00000100:00100000:1.0:1769074865.112515:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617037312:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074865.112611:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617037440:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 3289us (4203us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074865.112620:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1553 00000100:00100000:3.0:1769074865.119915:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007617037184:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 10482us (11560us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074865.119931:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2469 00000100:00100000:0.0:1769074865.125314:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617037568:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 13315us (16782us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074865.125331:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1554 00000100:00100000:1.0:1769074865.125976:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617037312:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 13458us (17480us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074865.125988:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2470 00010000:00010000:2.0:1769074865.565468:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074866.077245:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625246080:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074866.077318:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074866.077573:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625246208:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074866.077604:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074866.077724:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246080 00000100:00100000:0.0:1769074866.077730:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246080 00000100:00100000:0.0:1769074866.077855:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246208 00000100:00100000:0.0:1769074866.077859:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246208 00000100:00100000:3.0:1769074866.078591:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2735 00000100:00100000:3.0:1769074866.078608:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625246080:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074866.078644:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445363712 avail 1299124224 00000100:00100000:3.0:1769074866.078708:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625246080:12345-0@lo:41:osp-pre-0-1.0 Request processed in 105us (1389us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074866.078715:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2735 00000100:00100000:3.0:1769074866.078734:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2736 00000100:00100000:3.0:1769074866.078750:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625246208:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074866.078771:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:3.0:1769074866.078854:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625246208:12345-0@lo:41:osp-pre-1-0.0 Request processed in 114us (1248us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074866.078861:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2736 00000100:00100000:3.0:1769074866.078920:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625246208:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074866.079154:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625246080:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074866.589222:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074866.589248:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625246336:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074866.589314:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074866.589430:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246336 00000100:00100000:2.0:1769074866.589440:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246336 00000100:00100000:2.0:1769074866.589499:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1240 00000100:00100000:2.0:1769074866.589509:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625246336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074866.589538:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074866.589587:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625246336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074866.589594:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1240 00000100:00100000:2.0:1769074866.589633:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841306 gen 2 00000004:00080000:2.0:1769074866.589638:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5437b4e00, transno 21474841306 00000100:00100000:2.0:1769074866.589654:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625246336:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1769074866.589844:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1769074866.589943:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074866.590205:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074866.590216:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625246464:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074866.590254:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074866.590334:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246464 00000100:00100000:3.0:1769074866.590340:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246464 00000100:00100000:3.0:1769074866.590382:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1241 00000100:00100000:3.0:1769074866.590389:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b848050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625246464:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074866.590412:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 487044416 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074866.590445:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b848050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625246464:12345-0@lo:13:osp-pre-0-1.0 Request processed in 57us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074866.590452:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1241 00000100:00100000:1.0:1769074866.590885:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074866.590895:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625246592:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074866.590926:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074866.590956:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246592 00000100:00100000:1.0:1769074866.590961:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246592 00000100:00100000:1.0:1769074866.590987:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1242 00000100:00100000:1.0:1769074866.590994:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625246592:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074866.591012:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074866.591047:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625246592:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (120us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074866.591052:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1242 00000100:00100000:1.0:1769074866.591090:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625246592:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1769074866.591376:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074866.591380:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074866.591419:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625246464:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074867.105212:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1769074867.105240:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625246720:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074867.105313:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074867.105532:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246720 00000100:00100000:2.0:1769074867.105541:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246720 00000100:00100000:2.0:1769074867.105602:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1243 00000100:00100000:2.0:1769074867.105612:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625246720:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074867.105632:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074867.105672:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625246720:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074867.105678:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1243 00000100:00100000:3.0:1769074867.105857:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625246720:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1769074867.106138:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074867.614472:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074868.048216:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074868.048303:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617037824 00000100:00100000:1.0:1769074868.048315:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617037824 00000100:00100000:1.0:1769074868.048396:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13601 00000100:00100000:1.0:1769074868.048405:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:67421:x1855007617037824:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074868.048426:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074868.048448:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242400/0x7fe2323adfd559db lrc: 2/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8b7 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.048595:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048608:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048618:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048628:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048634:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048639:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048691:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559f0 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048697:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559f0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.048706:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559f0 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000004:00200000:1.0:1769074868.048726:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 0 00000100:00080000:0.0:1769074868.049054:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1769074868.049138:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049146:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049152:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049158:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049164:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074868.049168:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049175:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049187:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074868.049190:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074868.049195:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049200:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd559e2 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049213:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049223:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049229:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049234:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074868.049236:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049242:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049249:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b681697900) 00010000:00010000:1.0:1769074868.049251:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074868.049253:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049258:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd559e9 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.049298:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242400/0x7fe2323adfd559db lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8b7 expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.049312:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559f0 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8b7 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.049319:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697000) 00010000:00010000:1.0:1769074868.049321:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074868.049323:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242c00, rc 0) 00000100:00100000:1.0:1769074868.049388:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67421:x1855007617037824:12345-192.168.204.60@tcp:101:dd.0 Request processed in 983us (1180us total) trans 8589944123 rc 0/0 00000100:00100000:1.0:1769074868.049396:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13601 00000100:00100000:2.0:1769074868.070066:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074868.070165:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038080 00000100:00100000:1.0:1769074868.070176:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038080 00000100:00100000:1.0:1769074868.070236:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13602 00000100:00100000:1.0:1769074868.070245:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67421:x1855007617038080:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074868.070269:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074868.070272:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1769074868.070298:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd559f0 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8b7 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.070318:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697000) 00010000:00010000:1.0:1769074868.070328:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074868.070342:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242400/0x7fe2323adfd559fe lrc: 2/0,0 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8be expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.070390:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd55a05 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.070404:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd55a05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.070415:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd55a05 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00000040:00080000:1.0:1769074868.070694:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2471, 88 off211112 00000004:00080000:1.0:1769074868.070703:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1855912895698ns, next at 1845193472269ns, rc = 0 00000040:00080000:1.0:1769074868.070752:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2475, 88 off211704 00000004:00080000:1.0:1769074868.070759:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1855912951618ns, next at 1845220154830ns, rc = 0 00000001:00080000:0.0:1769074868.077666:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944124 is committed 00000004:00080000:0.0:1769074868.077680:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1971 committed 00000004:00080000:0.0:1769074868.077706:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1969 committed 00000100:00100000:2.0:1769074868.077938:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074868.077959:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625246848:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074868.078023:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074868.078115:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625246976:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1769074868.078138:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074868.078170:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246848 00000100:00100000:2.0:1769074868.078178:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246848 00000100:00100000:2.0:1769074868.078225:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625246976 00000100:00100000:2.0:1769074868.078229:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625246976 00000100:00100000:2.0:1769074868.078253:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14522 00000100:00100000:2.0:1769074868.078261:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625246848:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074868.078430:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625246848:12345-0@lo:2:osp-syn-1-0.0 Request processed in 170us (406us total) trans 17179874635 rc 0/0 00000100:00100000:2.0:1769074868.078439:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14522 00000100:00100000:2.0:1769074868.078455:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14523 00000100:00100000:2.0:1769074868.078461:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625246976:12345-0@lo:2:osp-syn-0-0.0 00000100:00080000:1.0:1769074868.078593:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074868.078598:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074868.078721:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625246976:12345-0@lo:2:osp-syn-0-0.0 Request processed in 246us (568us total) trans 21474841307 rc 0/0 00000100:00100000:2.0:1769074868.078728:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14523 00000004:00080000:2.0:1769074868.078766:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546c68e00/2, rc 0, transno 21474841307 00000100:00100000:2.0:1769074868.078771:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625246976:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1769074868.078781:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6656da680/2, rc 0, transno 17179874635 00000100:00100000:2.0:1769074868.078784:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656da680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625246848:0@lo:2:osp-syn-1-0.0 00010000:00010000:3.0:1769074868.079340:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd55a05 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8be expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074868.079355:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074868.079359:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074868.079374:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242400/0x7fe2323adfd559fe lrc: 0/0,0 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8be expref: 21 pid: 6513 timeout: 0 lvb_type: 3 00010000:00010000:3.0:1769074868.079395:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd55a05 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0xa/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8be expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074868.079423:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074868.079426:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074868.079429:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242e00, rc 0) 00000100:00100000:3.0:1769074868.079505:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:67421:x1855007617038080:12345-192.168.204.60@tcp:101:dd.0 Request processed in 9260us (9457us total) trans 8589944124 rc 0/0 00000100:00100000:3.0:1769074868.079514:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13602 00000100:00100000:3.0:1769074868.087389:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074868.087472:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038208 00000100:00100000:1.0:1769074868.087482:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038208 00000100:00100000:1.0:1769074868.087540:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13603 00000100:00100000:1.0:1769074868.087550:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:67421:x1855007617038208:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074868.087584:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074868.087613:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a0c lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8c5 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.087645:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd55a13 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.087659:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd55a13 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.087670:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd55a13 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT0 00010000:00010000:1.0:1769074868.087790:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a0c lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8c5 expref: 22 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.087853:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd55a13 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8c5 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.087862:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074868.087865:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1769074868.087868:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242000, rc 0) 00000100:00100000:1.0:1769074868.087948:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67421:x1855007617038208:12345-192.168.204.60@tcp:101:dd.0 Request processed in 398us (568us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074868.087956:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13603 00000100:00100000:0.0:1769074868.098872:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074868.098980:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038336 00000100:00100000:1.0:1769074868.098991:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038336 00000100:00100000:1.0:1769074868.099096:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8757 00000100:00100000:1.0:1769074868.099107:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617038336:12345-192.168.204.60@tcp:4:dd.0 00002000:00010000:1.0:1769074868.099159:0:8404:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074868.099169:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.099184:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074868.099186:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074868.099192:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.099205:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00080000:0.0:1769074868.104465:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1769074868.157884:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841308 is committed 00000001:00080000:0.0:1769074868.157905:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841307 is committed 00002000:00010000:1.0:1769074868.163932:0:8404:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:1.0:1769074868.163955:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb mtime from disk: 0 -> 1769074867 00002000:00010000:1.0:1769074868.163958:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb ctime from disk: 0 -> 1769074867 00002000:00010000:1.0:1769074868.163960:0:8404:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb blocks from disk: 0 -> 4096 00010000:00010000:1.0:1769074868.163977:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.163990:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.163999:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.164012:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074868.164016:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.164026:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.164048:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074868.164055:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549242c00/0x7fe2323adfd55a1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074868.164257:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617038336:12345-192.168.204.60@tcp:4:dd.0 Request processed in 65147us (65390us total) trans 21474841308 rc 0/0 00000100:00100000:1.0:1769074868.164268:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8757 00000100:00100000:3.0:1769074868.177616:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074868.178615:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038592 00000100:00100000:2.0:1769074868.178628:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038592 00000100:00100000:2.0:1769074868.178688:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4744 00000100:00100000:2.0:1769074868.178696:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67421:x1855007617038592:12345-192.168.204.60@tcp:35:dd.0 00000001:00080000:2.0:1769074868.179425:0:6516:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944125 00000100:00100000:2.0:1769074868.179671:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67421:x1855007617038592:12345-192.168.204.60@tcp:35:dd.0 Request processed in 972us (2062us total) trans 8589944126 rc 0/0 00000100:00100000:2.0:1769074868.179680:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4744 00000020:00080000:2.0:1769074868.179686:0:6516:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1769074868.643958:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074869.153417:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625247104:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153489:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153546:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625247616:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153572:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153592:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625248128:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153613:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153639:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625248640:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153657:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153677:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1769074869.153683:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625247232:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153717:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153734:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625247744:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153751:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153765:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625248256:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153781:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153822:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625247360:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153844:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153861:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625247872:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153878:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153892:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625248384:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153913:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153929:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1769074869.153935:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625247488:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.153955:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.153970:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625248000:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154002:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.154012:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625248512:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154031:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074869.154076:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247104 00000100:00100000:1.0:1769074869.154084:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247104 00000100:00100000:1.0:1769074869.154128:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1769074869.154137:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b666b8cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625247104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154185:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b666b8cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625247104:12345-0@lo:400:kworker.0 Request processed in 48us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154195:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1769074869.154227:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247616 00000100:00100000:1.0:1769074869.154231:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247616 00000100:00100000:1.0:1769074869.154264:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248384 00000100:00100000:1.0:1769074869.154267:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248384 00000100:00100000:1.0:1769074869.154284:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247488 00000100:00100000:1.0:1769074869.154287:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247488 00000100:00100000:1.0:1769074869.154303:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248512 00000100:00100000:1.0:1769074869.154306:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248512 00000100:00100000:1.0:1769074869.154324:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1555 00000100:00100000:1.0:1769074869.154329:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625247616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154358:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625247616:12345-0@lo:400:kworker.0 Request processed in 28us (785us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154363:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1555 00000100:00100000:1.0:1769074869.154374:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1556 00000100:00100000:1.0:1769074869.154379:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625248384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154406:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625248384:12345-0@lo:400:kworker.0 Request processed in 27us (492us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154412:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1556 00000100:00100000:1.0:1769074869.154422:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1557 00000100:00100000:1.0:1769074869.154428:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625247488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154451:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625247488:12345-0@lo:400:kworker.0 Request processed in 24us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154457:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1557 00000100:00100000:1.0:1769074869.154467:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1558 00000100:00100000:1.0:1769074869.154472:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625248512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154496:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625248512:12345-0@lo:400:kworker.0 Request processed in 24us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154501:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1558 00000100:00100000:1.0:1769074869.154516:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248128 00000100:00100000:1.0:1769074869.154520:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248128 00000100:00100000:1.0:1769074869.154538:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248640 00000100:00100000:1.0:1769074869.154541:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248640 00000100:00100000:1.0:1769074869.154551:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247744 00000100:00100000:1.0:1769074869.154554:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247744 00000100:00100000:1.0:1769074869.154564:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247360 00000100:00100000:1.0:1769074869.154567:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247360 00000100:00100000:1.0:1769074869.154576:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247872 00000100:00100000:1.0:1769074869.154578:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247872 00000100:00100000:1.0:1769074869.154588:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248000 00000100:00100000:1.0:1769074869.154591:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248000 00000100:00100000:1.0:1769074869.154603:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2471 00000100:00100000:1.0:1769074869.154608:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625248128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154637:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625248128:12345-0@lo:400:kworker.0 Request processed in 30us (1023us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154644:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2471 00000100:00100000:1.0:1769074869.154655:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2472 00000100:00100000:1.0:1769074869.154660:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625248640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154685:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625248640:12345-0@lo:400:kworker.0 Request processed in 24us (1028us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154701:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2472 00000100:00100000:1.0:1769074869.154713:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2473 00000100:00100000:1.0:1769074869.154719:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625247744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154745:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625247744:12345-0@lo:400:kworker.0 Request processed in 27us (993us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154750:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2473 00000100:00100000:1.0:1769074869.154761:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2474 00000100:00100000:1.0:1769074869.154766:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625247360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154791:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625247360:12345-0@lo:400:kworker.0 Request processed in 25us (946us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154796:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2474 00000100:00100000:1.0:1769074869.154829:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2475 00000100:00100000:1.0:1769074869.154836:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625247872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154861:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625247872:12345-0@lo:400:kworker.0 Request processed in 26us (982us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154866:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2475 00000100:00100000:1.0:1769074869.154877:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2476 00000100:00100000:1.0:1769074869.154883:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625248000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.154905:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625248000:12345-0@lo:400:kworker.0 Request processed in 23us (903us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.154910:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2476 00000100:00100000:1.0:1769074869.154928:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625247232 00000100:00100000:1.0:1769074869.154931:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625247232 00000100:00100000:1.0:1769074869.154953:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248256 00000100:00100000:1.0:1769074869.154956:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248256 00000100:00100000:1.0:1769074869.154973:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2737 00000100:00100000:1.0:1769074869.154979:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625247232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155011:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625247232:12345-0@lo:400:kworker.0 Request processed in 32us (1293us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.155016:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2737 00000100:00100000:1.0:1769074869.155028:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2738 00000100:00100000:1.0:1769074869.155034:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625248256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155063:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625248256:12345-0@lo:400:kworker.0 Request processed in 29us (1280us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074869.155069:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2738 00000100:00080000:1.0:1769074869.155119:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074869.155123:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074869.155126:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074869.155128:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074869.155174:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625247488:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155186:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625248000:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155196:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625248512:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155204:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625247360:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155214:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625247872:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155222:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841307 gen 2 00000004:00080000:1.0:1769074869.155227:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c68e00, transno 21474841307 00000100:00100000:1.0:1769074869.155239:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625248384:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155381:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625247232:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155391:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625247744:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155400:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625248256:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155409:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625247104:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155418:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625247616:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155428:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625248128:0@lo:400:kworker.0 00000100:00100000:1.0:1769074869.155437:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625248640:0@lo:400:kworker.0 00010000:00010000:0.0:1769074869.678348:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074870.236923:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074870.236960:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074870.237006:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074870.237029:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074870.237053:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074870.237081:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039104 00000100:00100000:0.0:1769074870.237092:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039104 00000100:00100000:0.0:1769074870.237143:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039232 00000100:00100000:0.0:1769074870.237147:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039232 00000100:00100000:0.0:1769074870.237170:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1559 00000100:00100000:0.0:1769074870.237180:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617039104:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074870.237257:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617039104:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 78us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074870.237264:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1559 00000100:00100000:0.0:1769074870.237281:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1560 00000100:00100000:0.0:1769074870.237288:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617039232:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074870.237315:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617039232:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 28us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074870.237321:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1560 00000100:00100000:0.0:1769074870.237338:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038720 00000100:00100000:0.0:1769074870.237341:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038720 00000100:00100000:0.0:1769074870.237368:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 381 00000100:00100000:0.0:1769074870.237375:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b666b8c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617038720:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074870.237400:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b666b8c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617038720:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 27us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074870.237406:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 381 00000100:00100000:0.0:1769074870.237437:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038848 00000100:00100000:0.0:1769074870.237440:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038848 00000100:00100000:0.0:1769074870.237467:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617038976 00000100:00100000:0.0:1769074870.237470:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617038976 00000100:00100000:0.0:1769074870.237493:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2477 00000100:00100000:0.0:1769074870.237499:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007617038848:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074870.237740:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007617038848:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 34us (623us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074870.237748:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2477 00000100:00100000:0.0:1769074870.237762:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2478 00000100:00100000:0.0:1769074870.237769:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617038976:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074870.237798:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617038976:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 31us (768us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074870.237827:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2478 00000100:00080000:0.0:1769074870.237849:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074870.237853:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074870.237856:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074870.237858:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1769074870.685059:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074871.197462:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625248768:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074871.197549:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074871.198155:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625248896:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074871.198186:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:0.0:1769074871.203504:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968371 is committed 00000100:00100000:2.0:1769074871.205943:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248768 00000100:00100000:2.0:1769074871.205959:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248768 00000100:00100000:2.0:1769074871.206020:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625248896 00000100:00100000:2.0:1769074871.206024:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625248896 00000100:00100000:2.0:1769074871.206051:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2739 00000100:00100000:2.0:1769074871.206061:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625248768:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074871.206098:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074871.206181:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625248768:12345-0@lo:41:osp-pre-1-0.0 Request processed in 120us (8631us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074871.206188:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2739 00000100:00100000:2.0:1769074871.206207:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2740 00000100:00100000:2.0:1769074871.206213:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625248896:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074871.206230:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:2.0:1769074871.206265:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625248896:12345-0@lo:41:osp-pre-0-1.0 Request processed in 52us (8079us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074871.206271:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2740 00000100:00100000:2.0:1769074871.206330:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625248896:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074871.206365:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968371 gen 2 00000004:00080000:2.0:1769074871.206371:0:3652:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968371 committed transno 4294968371 00000100:00100000:2.0:1769074871.206391:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625248768:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074871.709360:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625249024:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074871.709403:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074871.709417:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625249152:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074871.709429:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074871.709631:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249024 00000100:00100000:0.0:1769074871.709639:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249024 00000100:00100000:0.0:1769074871.709722:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1244 00000100:00100000:0.0:1769074871.709731:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625249024:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074871.709760:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074871.710269:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074871.710284:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625249280:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074871.710319:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074871.710882:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249152 00000100:00100000:2.0:1769074871.710890:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249152 00000100:00100000:2.0:1769074871.711177:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1245 00000100:00100000:2.0:1769074871.711187:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625249152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074871.711207:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00080000:1.0:1769074871.711339:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074871.712419:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625249152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1228us (2094us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074871.712430:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1245 00000100:00100000:3.0:1769074871.712442:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625249152:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074871.712644:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625249024:12345-0@lo:13:osp-pre-0-1.0 Request processed in 2913us (3213us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074871.712661:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1244 00000100:00100000:2.0:1769074871.712803:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625249024:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074871.712835:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1769074871.712876:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074871.712901:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249280 00000100:00100000:3.0:1769074871.712907:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249280 00000100:00100000:3.0:1769074871.712999:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1246 00000100:00100000:3.0:1769074871.713008:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625249280:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769074871.713033:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074871.713087:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625249280:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074871.713093:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1246 00000100:00100000:0.0:1769074871.713138:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625249280:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1769074871.713750:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074872.221267:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074872.221291:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625249408:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074872.221356:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074872.221667:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249408 00000100:00100000:0.0:1769074872.221676:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249408 00000100:00100000:0.0:1769074872.221744:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1247 00000100:00100000:0.0:1769074872.221754:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625249408:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074872.221777:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074872.221853:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625249408:12345-0@lo:13:osp-pre-1-1.0 Request processed in 101us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074872.221861:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1247 00000100:00080000:1.0:1769074872.221930:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074872.222164:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625249408:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1769074872.735401:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074873.762496:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074874.269910:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625249664:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.269987:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.270051:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625250176:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270077:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.270095:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625250688:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270115:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.270138:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1769074874.270161:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625249792:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270185:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.270201:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625250304:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270219:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.270234:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625250816:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270254:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.270282:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249664 00000100:00100000:0.0:1769074874.270290:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249664 00000100:00100000:0.0:1769074874.270327:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250688 00000100:00100000:0.0:1769074874.270330:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250688 00000100:00100000:0.0:1769074874.270348:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2741 00000100:00100000:0.0:1769074874.270356:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625249664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270412:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625249664:12345-0@lo:400:kworker.0 Request processed in 55us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074874.270419:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2741 00000100:00100000:0.0:1769074874.270435:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2742 00000100:00100000:0.0:1769074874.270441:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625250688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.270467:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625250688:12345-0@lo:400:kworker.0 Request processed in 27us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074874.270473:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2742 00000100:00100000:1.0:1769074874.270995:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625249920:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.271027:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074874.271264:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625249536:0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.271298:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074874.271350:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250176 00000100:00100000:0.0:1769074874.271356:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250176 00000100:00100000:3.0:1769074874.273399:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249792 00000100:00100000:3.0:1769074874.273411:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249792 00000100:00100000:1.0:1769074874.273507:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fa9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625250432:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.273564:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074874.273593:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625250944:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.273644:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1769074874.273686:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074874.273888:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250816 00000100:00100000:1.0:1769074874.273893:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250816 00000100:00100000:1.0:1769074874.273934:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249920 00000100:00100000:1.0:1769074874.273937:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249920 00000100:00100000:1.0:1769074874.273957:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250944 00000100:00100000:1.0:1769074874.273960:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250944 00000100:00100000:1.0:1769074874.273979:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1561 00000100:00100000:1.0:1769074874.273987:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625250816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274036:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625250816:12345-0@lo:400:kworker.0 Request processed in 50us (3780us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074874.274045:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1561 00000100:00100000:1.0:1769074874.274062:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1562 00000100:00100000:1.0:1769074874.274068:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625249920:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274100:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625249920:12345-0@lo:400:kworker.0 Request processed in 33us (3072us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074874.274106:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1562 00000100:00100000:1.0:1769074874.274117:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1563 00000100:00100000:1.0:1769074874.274122:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625250944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274165:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625250944:12345-0@lo:400:kworker.0 Request processed in 42us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074874.274171:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1563 00000100:00080000:1.0:1769074874.274213:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074874.274240:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074874.274271:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625250816:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274284:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625249664:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274295:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625250688:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274325:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625249920:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274335:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625250944:0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.274384:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250304 00000100:00100000:1.0:1769074874.274389:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250304 00000100:00100000:2.0:1769074874.274499:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625249536 00000100:00100000:2.0:1769074874.274504:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625249536 00000100:00100000:2.0:1769074874.274557:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1769074874.274564:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b666b8c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625249536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.274600:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b666b8c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625249536:12345-0@lo:400:kworker.0 Request processed in 36us (3301us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074874.274618:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1769074874.274711:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625249536:0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.274720:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625250048:0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.274747:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074874.274765:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625250560:0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.274786:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074874.274804:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625251072:0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.274840:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074874.274867:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250432 00000100:00100000:2.0:1769074874.274871:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250432 00000100:00100000:0.0:1769074874.281896:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250560 00000100:00100000:0.0:1769074874.281911:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250560 00000100:00100000:3.0:1769074874.282006:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251072 00000100:00100000:3.0:1769074874.282011:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251072 00000100:00100000:3.0:1769074874.282049:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2479 00000100:00100000:3.0:1769074874.282059:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625250176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282186:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625250176:12345-0@lo:400:kworker.0 Request processed in 128us (12106us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074874.282193:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2479 00000100:00100000:3.0:1769074874.282211:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2480 00000100:00100000:3.0:1769074874.282217:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625249792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282247:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625249792:12345-0@lo:400:kworker.0 Request processed in 31us (12062us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074874.282252:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2480 00000100:00100000:3.0:1769074874.282264:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2481 00000100:00100000:3.0:1769074874.282269:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625251072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282303:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625251072:12345-0@lo:400:kworker.0 Request processed in 34us (7462us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074874.282309:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2481 00000100:00100000:3.0:1769074874.282361:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625249792:0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282376:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625250176:0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282424:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625250048 00000100:00100000:3.0:1769074874.282428:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625250048 00000100:00100000:3.0:1769074874.282466:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1564 00000100:00100000:3.0:1769074874.282472:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625250048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282517:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625250048:12345-0@lo:400:kworker.0 Request processed in 45us (7769us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074874.282523:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1564 00000100:00100000:3.0:1769074874.282552:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625250048:0@lo:400:kworker.0 00000100:00100000:3.0:1769074874.282565:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625251072:0@lo:400:kworker.0 00000100:00080000:3.0:1769074874.282601:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074874.282616:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074874.282618:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074874.284154:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2482 00000100:00100000:1.0:1769074874.284170:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625250304:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074874.284248:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625250304:12345-0@lo:400:kworker.0 Request processed in 84us (14027us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074874.284256:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2482 00000100:00100000:1.0:1769074874.284317:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625250304:0@lo:400:kworker.0 00000001:00080000:1.0:1769074874.284497:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874635 is committed 00000100:00100000:2.0:1769074874.284842:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2483 00000100:00100000:2.0:1769074874.284854:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625250432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074874.284899:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625250432:12345-0@lo:400:kworker.0 Request processed in 49us (11334us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074874.284904:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2483 00000001:00080000:2.0:1769074874.285108:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944126 is committed 00000100:00100000:1.0:1769074874.285344:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fa9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625250432:0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.287904:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2484 00000100:00100000:0.0:1769074874.287929:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625250560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074874.288037:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625250560:12345-0@lo:400:kworker.0 Request processed in 121us (13249us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074874.288044:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2484 00000100:00100000:2.0:1769074874.288153:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b542fabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625250560:0@lo:400:kworker.0 00010000:00010000:0.0:1769074874.782010:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074875.338900:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074875.339038:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074875.339082:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074875.339105:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074875.339169:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039744 00000100:00100000:3.0:1769074875.339180:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039744 00000100:00100000:3.0:1769074875.339247:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1565 00000100:00100000:3.0:1769074875.339256:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617039744:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074875.339331:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617039744:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 75us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074875.339339:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1565 00000100:00080000:3.0:1769074875.339366:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074875.339541:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039360 00000100:00100000:1.0:1769074875.339547:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039360 00000100:00100000:1.0:1769074875.339594:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 383 00000100:00100000:1.0:1769074875.339603:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b666b8f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617039360:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074875.339641:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b666b8f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617039360:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 39us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074875.339647:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 383 00000100:00100000:0.0:1769074875.351988:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074875.352310:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039872 00000100:00100000:3.0:1769074875.352321:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039872 00000100:00100000:3.0:1769074875.352412:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1566 00000100:00100000:3.0:1769074875.352423:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617039872:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074875.352498:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617039872:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 77us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074875.352506:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1566 00000100:00080000:0.0:1769074875.353956:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074875.358682:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039488 00000100:00100000:2.0:1769074875.358699:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039488 00000100:00080000:0.0:1769074875.358957:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074875.359323:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617039616 00000100:00100000:2.0:1769074875.359328:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617039616 00000100:00080000:0.0:1769074875.359399:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074875.359674:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2485 00000100:00100000:3.0:1769074875.359688:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007617039488:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074875.359788:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2406:x1855007617039488:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 103us (20898us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074875.359796:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2485 00000100:00100000:3.0:1769074875.359834:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2486 00000100:00100000:3.0:1769074875.359841:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617039616:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074875.359868:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617039616:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 28us (20762us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074875.359874:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2486 00010000:00010000:1.0:1769074875.805367:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074876.317302:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074876.317327:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625251200:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074876.317414:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074876.317720:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251200 00000100:00100000:2.0:1769074876.317730:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251200 00000100:00100000:2.0:1769074876.317790:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2743 00000100:00100000:2.0:1769074876.317799:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625251200:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074876.317852:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074876.317913:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625251200:12345-0@lo:41:osp-pre-1-0.0 Request processed in 115us (500us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074876.317921:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2743 00000100:00100000:1.0:1769074876.318020:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625251200:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074876.318902:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625251328:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074876.318944:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074876.319031:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251328 00000100:00100000:2.0:1769074876.319038:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251328 00000100:00100000:2.0:1769074876.319070:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2744 00000100:00100000:2.0:1769074876.319077:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625251328:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074876.319102:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:2.0:1769074876.320367:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625251328:12345-0@lo:41:osp-pre-0-1.0 Request processed in 1284us (1416us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074876.320378:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2744 00000100:00100000:1.0:1769074876.330473:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b544758380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625251328:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074876.829357:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:2.0:1769074876.829357:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074876.829379:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625251456:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074876.829467:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074876.829536:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625251584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074876.829571:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074876.829594:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251456 00000100:00100000:1.0:1769074876.829602:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251456 00000100:00100000:1.0:1769074876.829660:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251584 00000100:00100000:1.0:1769074876.829663:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251584 00000100:00100000:1.0:1769074876.829684:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1248 00000100:00100000:1.0:1769074876.829693:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625251456:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074876.829720:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074876.829764:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625251456:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074876.829771:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1248 00000100:00100000:1.0:1769074876.829790:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1249 00000100:00100000:1.0:1769074876.829795:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625251584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074876.829843:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074876.829872:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625251584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074876.829877:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1249 00000100:00080000:0.0:1769074876.830130:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074876.830135:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074876.830176:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874635 gen 2 00000004:00080000:0.0:1769074876.830182:0:3650:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6656da680, transno 17179874635 00000100:00100000:0.0:1769074876.830204:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625251584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074876.830219:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625251456:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074876.830485:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074876.830496:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625251712:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074876.830529:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074876.830613:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251712 00000100:00100000:2.0:1769074876.830618:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251712 00000100:00100000:2.0:1769074876.830670:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1250 00000100:00100000:2.0:1769074876.830678:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625251712:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074876.830691:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:2.0:1769074876.830738:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625251712:12345-0@lo:13:osp-pre-0-0.0 Request processed in 61us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074876.830745:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1250 00000100:00100000:3.0:1769074876.830786:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625251712:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1769074876.831146:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074877.341941:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074877.341966:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625251840:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074877.342050:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074877.342321:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251840 00000100:00100000:2.0:1769074877.342330:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251840 00000100:00100000:2.0:1769074877.342406:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1251 00000100:00100000:2.0:1769074877.342416:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625251840:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074877.342438:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074877.342490:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625251840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 75us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074877.342497:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1251 00000100:00100000:2.0:1769074877.342567:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625251840:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1769074877.343323:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074877.853152:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074878.877365:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074879.389321:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625252096:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389391:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389460:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625252608:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389488:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389504:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625253120:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389534:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389558:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:1.0:1769074879.389564:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625251968:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389597:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389617:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625252480:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389642:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389662:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625252992:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389681:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389697:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625253504:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389716:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074879.389747:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252480 00000100:00100000:1.0:1769074879.389756:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252480 00000100:00100000:1.0:1769074879.389838:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1567 00000100:00100000:1.0:1769074879.389848:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625252480:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389898:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625252480:12345-0@lo:400:kworker.0 Request processed in 51us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074879.389905:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1567 00000100:00100000:1.0:1769074879.389944:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625252480:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.389978:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252608 00000100:00100000:1.0:1769074879.389981:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252608 00000100:00100000:1.0:1769074879.390004:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252992 00000100:00100000:1.0:1769074879.390006:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252992 00000100:00100000:1.0:1769074879.390019:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625253504 00000100:00100000:1.0:1769074879.390021:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625253504 00000100:00100000:1.0:1769074879.390032:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2487 00000100:00100000:1.0:1769074879.390038:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625252608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.390069:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625252608:12345-0@lo:400:kworker.0 Request processed in 31us (581us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074879.390075:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2487 00000100:00100000:1.0:1769074879.390085:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2488 00000100:00100000:1.0:1769074879.390090:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625252992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.390114:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fa9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625252992:12345-0@lo:400:kworker.0 Request processed in 24us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074879.390120:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2488 00000100:00100000:1.0:1769074879.390129:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2489 00000100:00100000:1.0:1769074879.390135:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625253504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.390158:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625253504:12345-0@lo:400:kworker.0 Request processed in 24us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074879.390164:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2489 00000100:00100000:1.0:1769074879.390185:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625252992:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.390194:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625253504:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.390201:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625252608:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390455:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b424a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625252352:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390488:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074879.390516:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625252864:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390548:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074879.390561:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625253376:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390579:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074879.390593:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1769074879.390598:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625252224:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390614:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074879.390628:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625252736:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390644:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074879.390653:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625253248:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390670:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074879.390696:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252864 00000100:00100000:3.0:1769074879.390701:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252864 00000100:00100000:3.0:1769074879.390724:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252224 00000100:00100000:3.0:1769074879.390726:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252224 00000100:00100000:3.0:1769074879.390737:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252736 00000100:00100000:3.0:1769074879.390739:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252736 00000100:00100000:3.0:1769074879.390751:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2490 00000100:00100000:3.0:1769074879.390757:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625252864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390789:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625252864:12345-0@lo:400:kworker.0 Request processed in 32us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.390795:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2490 00000100:00100000:3.0:1769074879.390834:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2491 00000100:00100000:3.0:1769074879.390839:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625252224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390874:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625252224:12345-0@lo:400:kworker.0 Request processed in 35us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.390880:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2491 00000100:00100000:3.0:1769074879.390890:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2492 00000100:00100000:3.0:1769074879.390896:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625252736:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.390919:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625252736:12345-0@lo:400:kworker.0 Request processed in 24us (274us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.390924:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2492 00000100:00100000:3.0:1769074879.390939:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252096 00000100:00100000:3.0:1769074879.390942:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252096 00000100:00100000:3.0:1769074879.390960:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625253120 00000100:00100000:3.0:1769074879.390971:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625253120 00000100:00100000:3.0:1769074879.390987:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2745 00000100:00100000:3.0:1769074879.390991:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625252096:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391019:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625252096:12345-0@lo:400:kworker.0 Request processed in 27us (1628us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.391025:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2745 00000100:00100000:3.0:1769074879.391034:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2746 00000100:00100000:3.0:1769074879.391039:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625253120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391060:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625253120:12345-0@lo:400:kworker.0 Request processed in 21us (1525us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.391065:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2746 00000100:00100000:3.0:1769074879.391081:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625251968 00000100:00100000:3.0:1769074879.391084:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625251968 00000100:00100000:3.0:1769074879.391101:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1769074879.391106:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625251968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391130:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625251968:12345-0@lo:400:kworker.0 Request processed in 24us (1533us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.391136:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1769074879.391162:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625252352 00000100:00100000:3.0:1769074879.391166:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625252352 00000100:00100000:3.0:1769074879.391186:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625253376 00000100:00100000:3.0:1769074879.391189:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625253376 00000100:00100000:3.0:1769074879.391202:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625253248 00000100:00100000:3.0:1769074879.391205:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625253248 00000100:00100000:3.0:1769074879.391219:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1568 00000100:00100000:3.0:1769074879.391224:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625252352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391250:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625252352:12345-0@lo:400:kworker.0 Request processed in 26us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.391256:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1568 00000100:00100000:3.0:1769074879.391265:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1569 00000100:00100000:3.0:1769074879.391270:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625253376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391297:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625253376:12345-0@lo:400:kworker.0 Request processed in 28us (718us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.391302:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1569 00000100:00100000:3.0:1769074879.391311:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1570 00000100:00100000:3.0:1769074879.391316:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625253248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391335:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625253248:12345-0@lo:400:kworker.0 Request processed in 20us (665us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074879.391341:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1570 00000100:00100000:3.0:1769074879.391377:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625252224:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391386:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b424000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625252736:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391395:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625253248:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391402:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b424a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625252352:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391410:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625252864:0@lo:400:kworker.0 00000100:00100000:3.0:1769074879.391418:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625253376:0@lo:400:kworker.0 00000100:00080000:0.0:1769074879.392563:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074879.392571:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074879.392574:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074879.392577:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074879.392713:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625251968:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.392730:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625252096:0@lo:400:kworker.0 00000100:00100000:1.0:1769074879.392741:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625253120:0@lo:400:kworker.0 00010000:00010000:2.0:1769074879.901022:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074880.452664:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074880.452706:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074880.452738:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074880.452770:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074880.452794:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074880.452850:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617040256 00000100:00100000:0.0:1769074880.452858:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617040256 00000100:00100000:0.0:1769074880.452906:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617040128 00000100:00100000:0.0:1769074880.452908:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617040128 00000100:00100000:0.0:1769074880.452926:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2493 00000100:00100000:0.0:1769074880.452935:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617040256:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074880.453018:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617040256:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 83us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074880.453025:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2493 00000100:00100000:0.0:1769074880.453041:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2494 00000100:00100000:0.0:1769074880.453047:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007617040128:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074880.453070:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:2408:x1855007617040128:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 23us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074880.453075:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2494 00000100:00100000:0.0:1769074880.453090:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617040512 00000100:00100000:0.0:1769074880.453093:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617040512 00000100:00100000:3.0:1769074880.453535:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617040384 00000100:00100000:3.0:1769074880.453540:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617040384 00000100:00100000:3.0:1769074880.453575:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1571 00000100:00100000:3.0:1769074880.453581:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617040384:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074880.453614:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617040384:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 33us (819us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074880.453620:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1571 00000100:00080000:3.0:1769074880.453657:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074880.453661:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074880.453663:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074880.453910:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1572 00000100:00100000:0.0:1769074880.453917:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617040512:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074880.453959:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617040512:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 43us (1220us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074880.453966:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1572 00000100:00080000:3.0:1769074880.454292:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074880.454343:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617040000 00000100:00100000:1.0:1769074880.454349:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617040000 00000100:00100000:1.0:1769074880.454391:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 385 00000100:00100000:1.0:1769074880.454397:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617040000:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074880.454432:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617040000:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 36us (1778us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074880.454438:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 385 00010000:00010000:1.0:1769074880.925876:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074881.325701:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074881.328625:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617040768 00000100:00100000:2.0:1769074881.328641:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617040768 00000100:00100000:2.0:1769074881.328835:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13604 00000100:00100000:2.0:1769074881.328846:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67444:x1855007617040768:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074881.328877:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074881.328909:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a21 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8cc expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074881.328986:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329000:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329014:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329026:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329033:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329039:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329140:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329147:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329152:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329158:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329165:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074881.329169:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329176:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329188:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa100) 00010000:00010000:2.0:1769074881.329191:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074881.329196:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329201:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a28 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329215:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329370:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329375:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329381:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074881.329383:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329391:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329398:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effac00) 00010000:00010000:2.0:1769074881.329401:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074881.329404:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329409:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a2f lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.329444:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a21 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8cc expref: 22 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074881.329454:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa100) 00010000:00010000:2.0:1769074881.329457:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074881.329460:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a21 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8cc expref: 22 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074881.329471:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074881.332283:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67444:x1855007617040768:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 3428us (6582us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074881.332297:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13604 00000100:00100000:0.0:1769074881.336610:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.336694:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617041024 00000100:00100000:3.0:1769074881.336704:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617041024 00000100:00100000:3.0:1769074881.336768:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13605 00000100:00100000:3.0:1769074881.336779:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67444:x1855007617041024:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.336835:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.336863:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a36 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8d3 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1769074881.337034:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x209:0x0]: returning size 4096 00010000:00010000:3.0:1769074881.337731:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a36 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8d3 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.337745:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074881.337748:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074881.337753:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a36 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8d3 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.337770:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28c00, rc 0) 00000100:00100000:3.0:1769074881.337884:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67444:x1855007617041024:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1101us (1279us total) trans 8589944127 rc 301/301 00000100:00100000:3.0:1769074881.337892:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13605 00000100:00100000:0.0:1769074881.345797:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074881.346427:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617041152 00000100:00100000:2.0:1769074881.346439:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617041152 00000100:00100000:2.0:1769074881.346506:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13606 00000100:00100000:2.0:1769074881.346517:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67444:x1855007617041152:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074881.346602:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346619:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346631:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346646:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346654:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346659:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346735:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346744:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.346751:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074881.346844:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00000004:00200000:2.0:1769074881.346849:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074881.346865:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349696:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349717:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349724:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349731:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349741:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074881.349746:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349755:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349773:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa100) 00010000:00010000:2.0:1769074881.349776:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074881.349781:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349788:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55a44 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349832:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349839:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349845:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349851:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074881.349853:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349859:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.349866:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effac00) 00010000:00010000:2.0:1769074881.349875:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074881.349879:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352568:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55a4b lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352601:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352608:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352615:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352620:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074881.352624:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352631:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352647:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074881.352649:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074881.352654:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074881.352659:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a52 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:2.0:1769074881.352755:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67444:x1855007617041152:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 6235us (6973us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074881.352766:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13606 00000100:00100000:0.0:1769074881.359139:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.359315:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617041408 00000100:00100000:3.0:1769074881.359327:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617041408 00000100:00100000:3.0:1769074881.359385:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4745 00000100:00100000:3.0:1769074881.359394:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67444:x1855007617041408:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:3.0:1769074881.359632:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67444:x1855007617041408:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 238us (503us total) trans 8589944128 rc 0/0 00000100:00100000:3.0:1769074881.359640:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4745 00000100:00100000:0.0:1769074881.413210:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074881.413386:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617041664 00000100:00100000:2.0:1769074881.413399:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617041664 00000100:00100000:2.0:1769074881.413468:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13607 00000100:00100000:2.0:1769074881.414134:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67448:x1855007617041664:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074881.414162:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074881.414186:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a59 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8da expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.414933:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.414950:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.414963:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.414978:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.414984:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.414990:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415097:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415105:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415111:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415117:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415122:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.415127:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415134:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415145:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effac00) 00010000:00010000:0.0:1769074881.415148:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.415154:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415159:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a60 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415173:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415178:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415183:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415188:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.415191:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415196:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415203:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b67a317700) 00010000:00010000:0.0:1769074881.415206:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.415208:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415213:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a67 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.415252:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a59 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8da expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.415263:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effac00) 00010000:00010000:0.0:1769074881.415266:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074881.415269:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55a59 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8da expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.415282:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88800, rc 0) 00000100:00100000:0.0:1769074881.415592:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67448:x1855007617041664:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 2114us (2392us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074881.415602:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13607 00000100:00100000:1.0:1769074881.435994:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.437294:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617041920 00000100:00100000:3.0:1769074881.437307:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617041920 00000100:00100000:3.0:1769074881.437379:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13608 00000100:00100000:3.0:1769074881.437389:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67448:x1855007617041920:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.437409:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.437431:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a6e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8e1 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.437683:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55a75 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x209:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.437697:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55a75 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x209:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.437708:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55a75 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x209:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074881.437719:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x209:0x0]: returning size 4096 00010000:00010000:3.0:1769074881.437886:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a6e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8e1 expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.437905:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29800/0x7fe2323adfd55a75 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x209:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8e1 expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.437914:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x209:0x0].0x0 (ffff89b64effa500) 00010000:00010000:3.0:1769074881.437917:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1769074881.437921:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29800, rc 0) 00000100:00100000:3.0:1769074881.437991:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67448:x1855007617041920:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 603us (2008us total) trans 8589944129 rc 0/0 00000100:00100000:3.0:1769074881.437999:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13608 00000100:00100000:0.0:1769074881.439460:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625253632:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074881.439535:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074881.441762:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074881.441785:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625253760:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074881.441878:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074881.442151:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625253632 00000100:00100000:2.0:1769074881.442161:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625253632 00000100:00100000:2.0:1769074881.442208:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625253760 00000100:00100000:2.0:1769074881.442211:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625253760 00000100:00100000:2.0:1769074881.442229:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2747 00000100:00100000:2.0:1769074881.442237:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625253632:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074881.442271:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074881.442323:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625253632:12345-0@lo:41:osp-pre-1-0.0 Request processed in 86us (2787us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074881.442331:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2747 00000100:00100000:2.0:1769074881.442349:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2748 00000100:00100000:2.0:1769074881.442355:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625253760:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074881.442372:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:2.0:1769074881.442404:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625253760:12345-0@lo:41:osp-pre-0-1.0 Request processed in 49us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074881.442409:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2748 00000100:00100000:2.0:1769074881.442451:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625253760:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074881.442494:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625253632:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074881.448137:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074881.448370:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617042048 00000100:00100000:0.0:1769074881.448381:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617042048 00000100:00100000:0.0:1769074881.448444:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13609 00000100:00100000:0.0:1769074881.448455:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67448:x1855007617042048:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074881.448559:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448576:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448588:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448599:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448606:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448612:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448687:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448696:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448703:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074881.448747:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00000004:00200000:0.0:1769074881.448752:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074881.448765:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448779:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448785:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448791:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448797:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448802:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.448851:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448859:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448871:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b67a317700) 00010000:00010000:0.0:1769074881.448874:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.448879:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448884:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55a83 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448902:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448907:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448912:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448917:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.448920:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448924:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448930:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b67a317300) 00010000:00010000:0.0:1769074881.448932:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.448935:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448939:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55a8a lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448948:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448953:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448958:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448963:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.448965:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448971:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448977:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:0.0:1769074881.448978:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.448981:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.448985:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55a91 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:0.0:1769074881.449047:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67448:x1855007617042048:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 594us (927us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074881.449055:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13609 00000100:00100000:2.0:1769074881.522263:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.522466:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617042304 00000100:00100000:3.0:1769074881.522477:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617042304 00000100:00100000:3.0:1769074881.522614:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13610 00000100:00100000:3.0:1769074881.522624:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67455:x1855007617042304:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.522646:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.522669:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a98 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8e8 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.522751:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522768:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522778:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522790:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522797:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522802:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522932:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522940:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522945:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522951:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522956:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.522960:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522966:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522981:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19de00) 00010000:00010000:3.0:1769074881.522984:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.522989:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.522994:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55a9f lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523007:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523013:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523018:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523023:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.523025:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523031:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523037:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19db00) 00010000:00010000:3.0:1769074881.523040:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.523042:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523048:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55aa6 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.523078:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a98 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8e8 expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.523087:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19de00) 00010000:00010000:3.0:1769074881.523090:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074881.523093:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55a98 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8e8 expref: 26 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.523102:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28c00, rc 0) 00000100:00100000:3.0:1769074881.523170:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67455:x1855007617042304:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 547us (920us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074881.523177:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13610 00000100:00100000:1.0:1769074881.540965:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.541104:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617042432 00000100:00100000:3.0:1769074881.541115:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617042432 00000100:00100000:3.0:1769074881.541174:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13611 00000100:00100000:3.0:1769074881.541184:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67455:x1855007617042432:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074881.541260:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541277:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541288:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541300:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541307:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541314:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541384:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541393:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541399:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074881.541441:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00000004:00200000:3.0:1769074881.541446:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074881.541459:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541473:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541479:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541497:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541512:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541518:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.541522:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541529:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541540:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19de00) 00010000:00010000:3.0:1769074881.541544:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.541549:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541556:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aad lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541572:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541577:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541582:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541587:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.541589:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541594:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541601:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19db00) 00010000:00010000:3.0:1769074881.541603:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.541605:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541610:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ab4 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541619:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541625:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541630:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541634:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.541637:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541643:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541650:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074881.541652:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.541655:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.541660:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55abb lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074881.541741:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67455:x1855007617042432:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 558us (787us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.541750:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13611 00000100:00100000:1.0:1769074881.579509:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074881.579616:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617042688 00000100:00100000:0.0:1769074881.579626:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617042688 00000100:00100000:0.0:1769074881.579689:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13612 00000100:00100000:0.0:1769074881.579699:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67460:x1855007617042688:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074881.579719:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074881.579743:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55ac2 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8ef expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.579865:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579879:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579891:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579901:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579908:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579914:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579982:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579989:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.579996:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580001:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580007:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.580011:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580017:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580032:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b67a317300) 00010000:00010000:0.0:1769074881.580035:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.580040:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580046:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ac9 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580059:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580065:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580071:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580075:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074881.580077:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580082:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580088:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b67a317700) 00010000:00010000:0.0:1769074881.580090:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074881.580093:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580098:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55ad0 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074881.580127:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55ac2 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8ef expref: 26 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.580137:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b67a317300) 00010000:00010000:0.0:1769074881.580139:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074881.580143:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55ac2 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8ef expref: 26 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.580153:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135800, rc 0) 00000100:00100000:0.0:1769074881.580204:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67460:x1855007617042688:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 507us (705us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074881.580212:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13612 00000100:00100000:2.0:1769074881.589285:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.589376:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617042816 00000100:00100000:3.0:1769074881.589387:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617042816 00000100:00100000:3.0:1769074881.589447:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13613 00000100:00100000:3.0:1769074881.589456:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67460:x1855007617042816:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074881.589543:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589560:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589576:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589588:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589594:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589600:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589670:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589678:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589684:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074881.589723:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00000004:00200000:3.0:1769074881.589727:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074881.589741:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589755:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589761:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589767:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589777:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589782:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.589785:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589792:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589804:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19db00) 00010000:00010000:3.0:1769074881.589857:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.589863:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589868:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55ad7 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589884:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589890:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589895:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589900:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.589903:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589909:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589919:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19de00) 00010000:00010000:3.0:1769074881.589922:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.589925:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589930:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55ade lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589940:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589946:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589952:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589957:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.589960:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589966:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589973:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074881.589975:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.589978:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.589982:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55ae5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074881.590048:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67460:x1855007617042816:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 592us (771us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.590058:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13613 00000100:00100000:2.0:1769074881.623414:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.623696:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617042944 00000100:00100000:3.0:1769074881.623707:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617042944 00000100:00100000:3.0:1769074881.623770:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13614 00000100:00100000:3.0:1769074881.623780:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:67463:x1855007617042944:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.623799:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.623855:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aec lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8f6 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.623914:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.623927:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.623938:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.623950:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.623956:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.623963:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624030:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624039:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624046:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074881.624085:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00000004:00200000:3.0:1769074881.624090:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074881.624108:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624119:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624414:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624423:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624430:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624436:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624442:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.624446:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624453:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624465:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19de00) 00010000:00010000:3.0:1769074881.624468:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.624473:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624489:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55af3 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624502:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624508:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624514:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624519:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074881.624521:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624526:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624532:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19db00) 00010000:00010000:3.0:1769074881.624534:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074881.624537:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624542:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55afa lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074881.624571:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55aec lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8f6 expref: 27 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.624586:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8f6 expref: 26 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.624597:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074881.624600:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074881.624603:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a29600, rc 0) 00000100:00100000:3.0:1769074881.625016:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:67463:x1855007617042944:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1235us (1609us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.625025:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13614 00000100:00100000:2.0:1769074881.651258:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.654644:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617043072 00000100:00100000:3.0:1769074881.654659:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617043072 00000100:00100000:3.0:1769074881.654716:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14524 00000100:00100000:3.0:1769074881.654724:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617043072:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.654742:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.654762:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c8fd expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074881.654783:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c9:0x0] initial LVB size: 2097152, mtime: 0x6971f0b3, atime: 0x0, ctime: 0x6971f0b3, blocks: 0x1000 00010000:00010000:3.0:1769074881.654799:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074881.654801:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074881.654824:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074881.654843:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074881.654855:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28c00, rc 0) 00000100:00100000:3.0:1769074881.654920:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617043072:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 196us (3671us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.654930:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14524 00000100:00100000:2.0:1769074881.668325:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074881.668609:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617043328 00000100:00100000:1.0:1769074881.668621:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617043328 00000100:00100000:1.0:1769074881.668684:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13615 00000100:00100000:1.0:1769074881.668694:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:67463:x1855007617043328:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074881.668714:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074881.668736:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242600/0x7fe2323adfd55b0f lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c904 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.668870:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b16 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074881.668884:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b16 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074881.668895:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b16 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074881.668908:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00010000:00010000:1.0:1769074881.669019:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242600/0x7fe2323adfd55b0f lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c904 expref: 29 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.669037:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b16 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c904 expref: 28 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.669044:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074881.669047:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074881.669051:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242800, rc 0) 00000100:00100000:1.0:1769074881.669103:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:67463:x1855007617043328:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 410us (787us total) trans 8589944130 rc 0/0 00000100:00100000:1.0:1769074881.669110:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13615 00000100:00100000:3.0:1769074881.674452:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074881.674557:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617043584 00000100:00100000:1.0:1769074881.674569:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617043584 00000100:00100000:1.0:1769074881.674630:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13616 00000100:00100000:1.0:1769074881.674640:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:67463:x1855007617043584:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074881.674665:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074881.674686:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55b24 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c90b expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.674792:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242200/0x7fe2323adfd55b2b lrc: 3/0,1 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074881.674846:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242200/0x7fe2323adfd55b2b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074881.674860:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242200/0x7fe2323adfd55b2b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:1.0:1769074881.674873:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00010000:00010000:1.0:1769074881.674975:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243a00/0x7fe2323adfd55b24 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c90b expref: 31 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.674993:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242200/0x7fe2323adfd55b2b lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c90b expref: 30 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.675002:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074881.675005:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1769074881.675009:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242200, rc 0) 00000100:00100000:1.0:1769074881.675069:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:67463:x1855007617043584:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 431us (627us total) trans 8589944131 rc 0/0 00000100:00100000:1.0:1769074881.675076:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13616 00000100:00100000:0.0:1769074881.676219:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074881.676331:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617043712 00000100:00100000:1.0:1769074881.676338:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617043712 00000100:00100000:1.0:1769074881.676396:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1046 00000100:00100000:1.0:1769074881.676405:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+30:2408:x1855007617043712:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074881.676416:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074881.676435:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b16 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c904 expref: 30 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.676452:0:8424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074881.676456:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074881.685062:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2408:x1855007617043712:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8646us (8836us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074881.685079:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1046 00000100:00100000:0.0:1769074881.685643:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074881.688201:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617043840 00000100:00100000:1.0:1769074881.688217:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617043840 00000100:00100000:1.0:1769074881.688293:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14525 00000100:00100000:1.0:1769074881.688302:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617043840:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:1.0:1769074881.688352:0:8396:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.688370:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074881.688383:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074881.688385:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074881.688388:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.688422:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074881.688436:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 9 pid: 28955 timeout: 1969 lvb_type: 1 00000100:00100000:1.0:1769074881.688451:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625253888:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074881.688561:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074881.696603:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8396:1855007625253888:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074881.696632:0:8396:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074881.702303:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.702353:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617043968 00000100:00100000:3.0:1769074881.702425:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617043968 00010000:00010000:3.0:1769074881.702464:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 10 pid: 28955 timeout: 1969 lvb_type: 1 00000100:00100000:3.0:1769074881.702542:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7226 00000100:00100000:3.0:1769074881.702552:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617043968:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074881.702568:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074881.702591:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 10 pid: 28955 timeout: 1969 lvb_type: 1 00010000:00010000:3.0:1769074881.702604:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 10 pid: 28955 timeout: 1969 lvb_type: 1 00010000:00010000:3.0:1769074881.702629:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b08 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8fd expref: 10 pid: 28955 timeout: 1969 lvb_type: 1 00010000:00010000:3.0:1769074881.702646:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074881.702648:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074881.702653:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702662:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702693:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074881.702742:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617043968:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 191us (450us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.702750:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7226 00010000:00010000:3.0:1769074881.702780:0:8396:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702788:0:8396:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702829:0:8396:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702838:0:8396:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702845:0:8396:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702852:0:8396:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243c00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074881.702856:0:8396:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702864:0:8396:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702886:0:8396:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.702892:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243c00/0x7fe2323adfd55b39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074881.702935:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617043840:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 14634us (17296us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.702943:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14525 00000100:00100000:3.0:1769074881.707465:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.707516:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044224 00000100:00100000:3.0:1769074881.707527:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044224 00000100:00100000:3.0:1769074881.707581:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13617 00000100:00100000:3.0:1769074881.707591:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:67463:x1855007617044224:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074881.707643:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1769074881.707670:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b01 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8f6 expref: 29 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.707690:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242e00/0x7fe2323adfd55a05 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8be expref: 28 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.707701:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074881.707706:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074881.707733:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.707747:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.707757:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00080000:3.0:1769074881.707960:0:6514:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2cc:0x0], index 1: 0 00000040:00080000:3.0:1769074881.708039:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2472, 88 off211200 00000004:00080000:3.0:1769074881.708048:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1869550240592ns, next at 1845193472269ns, rc = 0 00000040:00080000:3.0:1769074881.708087:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2476, 88 off211792 00000004:00080000:3.0:1769074881.708093:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1869550285560ns, next at 1845220154830ns, rc = 0 00000040:00080000:3.0:1769074881.708109:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2477, 88 off211880 00000004:00080000:3.0:1769074881.708113:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1869550306379ns, next at 1845220154830ns, rc = 0 00000001:00080000:0.0:1769074881.726624:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944132 is committed 00000004:00080000:0.0:1769074881.726638:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1973 committed 00000004:00080000:0.0:1769074881.726659:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1972 committed 00000004:00080000:0.0:1769074881.726662:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1970 committed 00000100:00100000:0.0:1769074881.726768:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625254016:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1769074881.726874:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074881.726930:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625254016 00000100:00100000:0.0:1769074881.726938:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625254016 00000100:00100000:0.0:1769074881.726988:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14526 00000100:00100000:0.0:1769074881.726996:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625254016:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1769074881.727390:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625254144:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074881.727425:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074881.727450:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074881.727455:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625254272:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074881.727473:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074881.727511:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625254144 00000100:00100000:1.0:1769074881.727515:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625254144 00000100:00100000:1.0:1769074881.727538:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625254272 00000100:00100000:1.0:1769074881.727541:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625254272 00000100:00100000:1.0:1769074881.727559:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14527 00000100:00100000:1.0:1769074881.727566:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625254144:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074881.727680:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625254144:12345-0@lo:2:osp-syn-1-0.0 Request processed in 113us (253us total) trans 17179874636 rc 0/0 00000100:00100000:1.0:1769074881.727687:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14527 00000100:00100000:1.0:1769074881.727706:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14528 00000100:00100000:1.0:1769074881.727712:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625254272:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074881.727773:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625254272:12345-0@lo:2:osp-syn-1-0.0 Request processed in 61us (299us total) trans 17179874637 rc 0/0 00000100:00100000:1.0:1769074881.727780:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14528 00000004:00080000:1.0:1769074881.727844:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d0e6680/2, rc 0, transno 17179874637 00000100:00100000:1.0:1769074881.727850:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625254272:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1769074881.727860:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d0e4a80/2, rc 0, transno 17179874636 00000100:00100000:1.0:1769074881.727863:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625254144:0@lo:2:osp-syn-1-0.0 00000004:00010000:3.0:1769074881.728304:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000a12b8ca2 reply state 00000000a0240d04 transno 8589944132 ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.728327:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.728336:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074881.728339:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:3.0:1769074881.728342:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00080000:3.0:1769074881.728429:0:6514:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1769074881.728445:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67463:x1855007617044224:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 20854us (20989us total) trans 8589944132 rc 0/0 00000100:00100000:3.0:1769074881.728453:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13617 00000100:00100000:0.0:1769074881.729078:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625254016:12345-0@lo:2:osp-syn-0-0.0 Request processed in 2079us (2200us total) trans 21474841309 rc 0/0 00000100:00100000:0.0:1769074881.729087:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14526 00000100:00080000:0.0:1769074881.729110:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074881.729115:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:1.0:1769074881.729344:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54b426a00/2, rc 0, transno 21474841309 00000100:00100000:1.0:1769074881.729351:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625254016:0@lo:2:osp-syn-0-0.0 00010000:00010000:2.0:1769074881.729781:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074881.729792:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074881.729799:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074881.729842:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b543a28800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074881.729847:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074881.729855:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074881.729868:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074881.729872:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074881.729876:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074881.729881:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28800/0x7fe2323adfd55b40 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000100:00100000:1.0:1769074881.742876:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074881.746377:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044352 00000100:00100000:1.0:1769074881.746394:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044352 00000100:00100000:1.0:1769074881.746470:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13618 00000100:00100000:1.0:1769074881.746495:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67463:x1855007617044352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074881.746520:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074881.746542:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c912 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.746608:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.746623:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.746644:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074881.746647:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074881.746651:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242800, rc 0) 00000100:00100000:1.0:1769074881.746722:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67463:x1855007617044352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 242us (3856us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074881.746729:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13618 00000100:00100000:3.0:1769074881.773788:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.773839:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044480 00000100:00100000:3.0:1769074881.773849:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044480 00000100:00100000:3.0:1769074881.773905:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13619 00000100:00100000:3.0:1769074881.773914:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67463:x1855007617044480:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.773936:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.773956:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b4e lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c919 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.774048:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b55 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.774063:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b55 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.774075:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b55 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074881.774101:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097153 00000004:00200000:3.0:1769074881.774106:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074881.774124:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b55 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074881.774152:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a29600/0x7fe2323adfd55b4e lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c919 expref: 25 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.774167:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b55 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c919 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074881.774176:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074881.774179:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074881.774183:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28c00, rc 0) 00000100:00100000:3.0:1769074881.774239:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67463:x1855007617044480:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 326us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.774246:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13619 00000100:00100000:3.0:1769074881.788628:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.788672:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044608 00000100:00100000:3.0:1769074881.788681:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044608 00000100:00100000:3.0:1769074881.788730:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14529 00000100:00100000:3.0:1769074881.788740:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617044608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074881.788760:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074881.788785:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c920 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074881.788827:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000401:0x2c9:0x0] initial LVB size: 2097152, mtime: 0x6971f0b3, atime: 0x0, ctime: 0x6971f0b3, blocks: 0x1000 00010000:00010000:3.0:1769074881.788847:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074881.788849:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074881.788862:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074881.788879:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 9 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074881.788891:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b543a28600, rc 0) 00000100:00100000:3.0:1769074881.788943:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617044608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 204us (325us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.788950:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14529 00000100:00100000:2.0:1769074881.857397:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074881.857432:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044736 00000100:00100000:2.0:1769074881.857442:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044736 00000100:00100000:2.0:1769074881.857506:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14530 00000100:00100000:2.0:1769074881.857516:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617044736:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:2.0:1769074881.857612:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617044736:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 97us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074881.857620:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14530 00000100:00100000:2.0:1769074881.871644:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074881.871863:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044864 00000100:00100000:3.0:1769074881.871875:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044864 00000100:00100000:3.0:1769074881.871932:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14531 00000100:00100000:3.0:1769074881.871942:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617044864:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:3.0:1769074881.872056:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617044864:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 115us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074881.872067:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14531 00000100:00100000:3.0:1769074881.878361:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074881.878504:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617044992 00000100:00100000:1.0:1769074881.878515:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617044992 00000100:00100000:1.0:1769074881.878586:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8758 00000100:00100000:1.0:1769074881.878596:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617044992:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:1.0:1769074881.878634:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.878652:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074881.878654:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074881.878660:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074881.878674:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952883:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952903:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952914:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952921:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074881.952926:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952935:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952953:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.952960:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243400/0x7fe2323adfd55b63 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->2097151] (req 0->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074881.953055:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617044992:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 74456us (74699us total) trans 0 rc 2097152/2097152 00000100:00100000:0.0:1769074881.953066:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8758 00000100:00100000:0.0:1769074881.956702:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625254400:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074881.956774:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074881.957014:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074881.957022:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625254528:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074881.957050:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1769074881.958700:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074881.958780:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625254400 00000100:00100000:1.0:1769074881.958787:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625254400 00000100:00100000:1.0:1769074881.958852:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625254528 00000100:00100000:1.0:1769074881.958855:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625254528 00000100:00100000:1.0:1769074881.958888:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1252 00000100:00100000:1.0:1769074881.958897:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625254400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074881.958927:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074881.959000:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625254400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (2226us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074881.959008:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1252 00000100:00100000:1.0:1769074881.959027:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1253 00000100:00100000:1.0:1769074881.959034:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625254528:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074881.959043:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074881.959065:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625254528:12345-0@lo:13:osp-pre-0-1.0 Request processed in 32us (2015us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074881.959071:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1253 00000100:00100000:1.0:1769074881.959142:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074881.959155:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625254656:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074881.959188:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074881.959218:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625254656 00000100:00100000:1.0:1769074881.959222:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625254656 00000100:00100000:1.0:1769074881.959239:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1254 00000100:00100000:1.0:1769074881.959245:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625254656:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074881.959263:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 37107008 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074881.959292:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625254656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 47us (104us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074881.959299:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1254 00000100:00080000:1.0:1769074881.959314:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074881.959351:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625254528:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074881.959367:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625254400:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074881.959399:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625254656:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074881.985349:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074881.989896:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617045120 00000100:00100000:0.0:1769074881.989911:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617045120 00000100:00100000:0.0:1769074881.989994:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8759 00000100:00100000:0.0:1769074881.990005:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617045120:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:0.0:1769074881.990057:0:8404:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2cb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1769074881.990067:0:8404:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.990082:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074881.990085:0:8404:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074881.990090:0:8404:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074881.990106:0:8404:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1769074882.077236:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874638 is committed 00000001:00080000:1.0:1769074882.077257:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874637 is committed 00002000:00010000:2.0:1769074882.078269:0:8404:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:2.0:1769074882.078276:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb mtime from disk: 0 -> 1769074881 00002000:00010000:2.0:1769074882.078279:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb ctime from disk: 0 -> 1769074881 00002000:00010000:2.0:1769074882.078281:0:8404:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:2.0:1769074882.078296:0:8404:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.078308:0:8404:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.078317:0:8404:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.078324:0:8404:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074882.078329:0:8404:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.078338:0:8404:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.078356:0:8404:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.078364:0:8404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55b6a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8404 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074882.078466:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617045120:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 88458us (93121us total) trans 17179874638 rc 0/0 00000100:00100000:2.0:1769074882.078488:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8759 00000100:00100000:1.0:1769074882.086332:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.099098:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617045248 00000100:00100000:2.0:1769074882.099116:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617045248 00000100:00080000:2.0:1769074882.099347:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074882.099416:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8760 00000100:00100000:2.0:1769074882.099426:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617045248:12345-192.168.204.60@tcp:4:lfs.0 00002000:00010000:2.0:1769074882.099488:0:28434:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2cc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1769074882.099498:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.099512:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074882.099515:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074882.099520:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.099533:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1769074882.146612:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874639 is committed 00002000:00010000:2.0:1769074882.148047:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:2.0:1769074882.148053:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb mtime from disk: 0 -> 1769074881 00002000:00010000:2.0:1769074882.148056:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb ctime from disk: 0 -> 1769074881 00002000:00010000:2.0:1769074882.148058:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:2.0:1769074882.148072:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.148089:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.148097:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.148104:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074882.148109:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.148117:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.148504:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.148513:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b71 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074882.148615:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617045248:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 49187us (62288us total) trans 17179874639 rc 0/0 00000100:00100000:2.0:1769074882.148626:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8760 00000100:00100000:0.0:1769074882.210173:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074882.210984:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617045504 00000100:00100000:3.0:1769074882.210997:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617045504 00000100:00080000:2.0:1769074882.211086:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074882.211227:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13620 00000100:00100000:3.0:1769074882.211239:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67463:x1855007617045504:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074882.211302:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074882.211335:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b543a28c00/0x7fe2323adfd55b55 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c919 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.211352:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074882.211357:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074882.211583:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,1 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.211597:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.211607:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000001:00080000:3.0:1769074882.211649:0:10283:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 8589944133 00000004:00010000:3.0:1769074882.211671:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000b0c484c6 reply state 00000000a93d8855 transno 8589944134 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.211682:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.211688:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074882.211692:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074882.211694:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769074882.211795:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:67463:x1855007617045504:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 558us (1631us total) trans 8589944134 rc 0/0 00000100:00100000:3.0:1769074882.211803:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13620 00000020:00080000:3.0:1769074882.211830:0:10283:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1769074882.225000:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074882.225165:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617045632 00000100:00100000:1.0:1769074882.225175:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617045632 00000100:00100000:1.0:1769074882.225232:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14532 00000100:00100000:1.0:1769074882.225241:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:67463:x1855007617045632:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074882.225259:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074882.225281:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c927 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074882.225314:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2cb:0x0] initial LVB size: 1048576, mtime: 0x6971f0c1, atime: 0x0, ctime: 0x6971f0c1, blocks: 0x800 00010000:00010000:1.0:1769074882.225334:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074882.225336:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074882.225344:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074882.225362:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074882.225381:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243800, rc 0) 00000100:00100000:1.0:1769074882.225433:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:67463:x1855007617045632:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 194us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074882.225440:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14532 00000100:00100000:1.0:1769074882.228015:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.232116:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617045760 00000100:00100000:2.0:1769074882.232586:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617045760 00000100:00100000:2.0:1769074882.232661:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14533 00000100:00100000:2.0:1769074882.232672:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:67463:x1855007617045760:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074882.232699:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074882.232728:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c92e expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074882.232752:0:28955:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x2cc:0x0] initial LVB size: 2097152, mtime: 0x6971f0c1, atime: 0x0, ctime: 0x6971f0c1, blocks: 0x800 00010000:00010000:2.0:1769074882.232772:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074882.232774:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074882.232783:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.232799:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 9 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.233014:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074882.233083:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:67463:x1855007617045760:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 412us (5077us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074882.233091:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14533 00000100:00100000:0.0:1769074882.238927:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.239001:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617045888 00000100:00100000:2.0:1769074882.239013:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617045888 00002000:00010000:2.0:1769074882.239055:0:28434:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/716 (2097152->18446744073709551615) 00002000:00100000:2.0:1769074882.239060:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427100 with x1855007617045888 ext(2097152->18446744073709551615) 00002000:00010000:2.0:1769074882.239073:0:28434:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b427100. 00000100:00100000:2.0:1769074882.239112:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8761 00000100:00100000:2.0:1769074882.239121:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617045888:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:2.0:1769074882.239299:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074882.239304:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb mtime from disk: 1769074881 -> 1769074881 00002000:00010000:2.0:1769074882.239306:0:28434:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074882.239308:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb ctime from disk: 1769074881 -> 1769074881 00002000:00010000:2.0:1769074882.239310:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:2.0:1769074882.239371:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2407:x1855007617045888:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 250us (455us total) trans 17179874640 rc 0/0 00000100:00100000:2.0:1769074882.239380:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8761 00002000:00010000:2.0:1769074882.239384:0:28434:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/716 (2097152->18446744073709551615) 00002000:00100000:2.0:1769074882.239386:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54b427100 with x1855007617045888 ext(2097152->18446744073709551615) 00002000:00010000:2.0:1769074882.239391:0:28434:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff89b54b427100. 00000100:00100000:0.0:1769074882.242902:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.246878:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617046144 00000100:00100000:2.0:1769074882.246893:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617046144 00000100:00100000:2.0:1769074882.246970:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13621 00000100:00100000:2.0:1769074882.246981:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:67463:x1855007617046144:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074882.247101:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,1 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.247120:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.247131:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00010000:2.0:1769074882.247245:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000004551b9f7 reply state 00000000625d70fb transno 8589944135 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.247259:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.247267:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074882.247270:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074882.247273:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074882.247388:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:67463:x1855007617046144:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 408us (4496us total) trans 8589944135 rc 0/0 00000100:00100000:2.0:1769074882.247396:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13621 00000100:00100000:0.0:1769074882.252612:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.253338:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617046272 00000100:00100000:2.0:1769074882.253349:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617046272 00000100:00100000:2.0:1769074882.253494:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14534 00000100:00100000:2.0:1769074882.253505:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:67463:x1855007617046272:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074882.253524:0:28955:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074882.253543:0:28955:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c935 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.253573:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 10 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074882.253581:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074882.253583:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074882.253587:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->10485759] (req 0->10485759) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.253622:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 10 pid: 28955 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074882.253636:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 10 pid: 28955 timeout: 1970 lvb_type: 1 00000100:00100000:2.0:1769074882.253649:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625254784:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074882.253740:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64133a780 going to sleep for 11 seconds 00000100:00080000:1.0:1769074882.256393:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074882.291909:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074882.292003:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617046400 00000100:00100000:1.0:1769074882.292014:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617046400 00010000:00010000:1.0:1769074882.292056:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 11 pid: 28955 timeout: 1970 lvb_type: 1 00000100:00100000:1.0:1769074882.292098:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7227 00000100:00100000:1.0:1769074882.292108:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617046400:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074882.292142:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074882.292163:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 11 pid: 28955 timeout: 1970 lvb_type: 1 00010000:00010000:1.0:1769074882.292178:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 11 pid: 28955 timeout: 1970 lvb_type: 1 00010000:00010000:1.0:1769074882.292205:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074882.292208:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074882.292215:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074882.292226:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50080000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074882.292264:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 11 pid: 28955 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074882.292279:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625254912:192.168.204.60@tcp:105: 00000100:00100000:1.0:1769074882.292366:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67a397a80 going to sleep for 11 seconds 00010000:00010000:2.0:1769074882.292710:0:28955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b543a28600/0x7fe2323adfd55b5c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c920 expref: 11 pid: 28955 timeout: 1970 lvb_type: 1 00000100:00100000:2.0:1769074882.292726:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:28955:1855007625254784:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074882.292749:0:28955:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 10 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.292759:0:28955:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89200, rc 0) 00000100:00100000:2.0:1769074882.292801:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:67463:x1855007617046272:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 39298us (40199us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074882.292853:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14534 00000100:00100000:1.0:1769074882.300522:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625254912:192.168.204.60@tcp:105: 00010000:00010000:1.0:1769074882.300555:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074882.300616:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617046400:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 8507us (8719us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074882.300627:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7227 00000100:00100000:0.0:1769074882.305090:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.305193:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617046528 00000100:00100000:2.0:1769074882.305206:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617046528 00000100:00100000:2.0:1769074882.305265:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14535 00000100:00100000:2.0:1769074882.305275:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617046528:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:2.0:1769074882.305435:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:2.0:1769074882.305439:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb mtime from disk: 1769074867 -> 1769074867 00002000:00010000:2.0:1769074882.305442:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb atime from disk: 0 -> 1769074855 00002000:00010000:2.0:1769074882.305444:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb ctime from disk: 1769074867 -> 1769074881 00002000:00010000:2.0:1769074882.305447:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:2.0:1769074882.305518:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617046528:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 244us (439us total) trans 21474841310 rc 0/0 00000100:00100000:2.0:1769074882.305526:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14535 00000100:00100000:0.0:1769074882.310753:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.310873:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617046784 00000100:00100000:2.0:1769074882.310884:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617046784 00000100:00100000:2.0:1769074882.310942:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4746 00000100:00100000:2.0:1769074882.310952:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:67463:x1855007617046784:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:2.0:1769074882.311015:0:15947:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x200000405:0x20a:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242200/0x7fe2323adfd55b2b lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c90b expref: 28 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.311062:0:15947:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.311076:0:15947:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 28 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.311086:0:15947:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 4/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.311120:0:15947:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 28 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.311133:0:15947:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 28 pid: 6514 timeout: 1970 lvb_type: 0 00000100:00100000:2.0:1769074882.311148:0:15947:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:15947:1855007625255040:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074882.311231:0:15947:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64133a780 going to sleep for 11 seconds 00000100:00100000:0.0:1769074882.315625:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074882.315727:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617046912 00000100:00100000:1.0:1769074882.315738:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617046912 00010000:00010000:1.0:1769074882.315777:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 29 pid: 6514 timeout: 1970 lvb_type: 0 00000100:00100000:1.0:1769074882.315861:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7228 00000100:00100000:1.0:1769074882.315871:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007617046912:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074882.315886:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074882.315893:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 5/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 29 pid: 6514 timeout: 1970 lvb_type: 0 00010000:00010000:1.0:1769074882.315907:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 4/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 29 pid: 6514 timeout: 1970 lvb_type: 0 00010000:00010000:1.0:1769074882.315929:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:1.0:1769074882.315935:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT0 00010000:00010000:1.0:1769074882.315946:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 15947 initiator: MDT0 00010000:00010000:1.0:1769074882.315954:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 15947 initiator: MDT0 00010000:00010000:1.0:1769074882.315972:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074882.316017:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:2409:x1855007617046912:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 148us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074882.316025:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7228 00010000:00010000:2.0:1769074882.318102:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242800/0x7fe2323adfd55b47 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c912 expref: 28 pid: 6514 timeout: 1970 lvb_type: 0 00000100:00100000:2.0:1769074882.318126:0:15947:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:15947:1855007625255040:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074882.318150:0:15947:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.318163:0:15947:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.318167:0:15947:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT0 00000040:00080000:2.0:1769074882.318401:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2473, 88 off211288 00000004:00080000:2.0:1769074882.318410:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1870160602812ns, next at 1845193472269ns, rc = 0 00000040:00080000:2.0:1769074882.318465:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2478, 88 off211968 00000004:00080000:2.0:1769074882.318482:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1870160675054ns, next at 1845220154830ns, rc = 0 00000040:00080000:2.0:1769074882.318509:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2479, 88 off212056 00000004:00080000:2.0:1769074882.318513:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1870160706234ns, next at 1845220154830ns, rc = 0 00000001:00080000:2.0:1769074882.322257:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944136 is committed 00000004:00080000:2.0:1769074882.322271:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1975 committed 00000004:00080000:2.0:1769074882.322297:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1974 committed 00000004:00080000:2.0:1769074882.322300:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1971 committed 00000004:00010000:2.0:1769074882.322375:0:15947:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000008b12b45b reply state 0000000089e2fc23 transno 8589944136 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322404:0:15947:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322413:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074882.322416:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:2.0:1769074882.322420:0:15947:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074882.322428:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074882.322430:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074882.322438:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242200/0x7fe2323adfd55b2b lrc: 0/0,0 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c90b expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00080000:2.0:1769074882.322577:0:15947:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1769074882.322592:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:67463:x1855007617046784:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 11640us (11848us total) trans 8589944136 rc 0/0 00000100:00100000:2.0:1769074882.322603:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4746 00010000:00010000:2.0:1769074882.322670:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322678:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322685:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322690:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.322695:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322702:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322711:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074882.322714:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074882.322717:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074882.322723:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55b9b lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT0 00000100:00100000:2.0:1769074882.323236:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625255168:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074882.323425:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625255424:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074882.323512:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625255296:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1769074882.323565:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1769074882.323660:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323673:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323679:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323685:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074882.323688:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323694:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323706:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:0.0:1769074882.323709:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074882.323713:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323719:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb600/0x7fe2323adfd55b78 lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.323736:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074882.323742:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074882.323748:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074882.323752:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074882.323754:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074882.323763:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074882.323770:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:0.0:1769074882.323772:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074882.323775:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074882.323779:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55b8d lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:1.0:1769074882.331299:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074882.331384:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255168 00000100:00100000:1.0:1769074882.331395:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255168 00000100:00100000:1.0:1769074882.331436:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255296 00000100:00100000:1.0:1769074882.331439:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255296 00000100:00100000:1.0:1769074882.331460:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14536 00000100:00100000:1.0:1769074882.331480:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625255168:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1769074882.331635:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074882.331640:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb mtime from disk: 1769074867 -> 1769074867 00002000:00010000:1.0:1769074882.331642:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb atime from disk: 1769074855 -> 1769074855 00002000:00010000:1.0:1769074882.331645:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb ctime from disk: 1769074881 -> 1769074881 00002000:00010000:1.0:1769074882.331647:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000401:0x2c9:0x0] updating lvb blocks from disk: 4096 -> 4096 00000100:00100000:1.0:1769074882.331703:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625255168:12345-0@lo:2:osp-syn-0-0.0 Request processed in 235us (8415us total) trans 21474841311 rc 0/0 00000100:00100000:1.0:1769074882.331710:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14536 00000100:00100000:1.0:1769074882.331728:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14537 00000100:00100000:1.0:1769074882.331734:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625255296:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074882.331797:0:28955:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb size from disk: 1048576 -> 1048576 00002000:00010000:1.0:1769074882.331800:0:28955:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb mtime from disk: 1769074881 -> 1769074881 00002000:00010000:1.0:1769074882.331803:0:28955:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074882.331846:0:28955:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb ctime from disk: 1769074881 -> 1769074881 00002000:00010000:1.0:1769074882.331849:0:28955:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2cb:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074882.331882:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625255296:12345-0@lo:2:osp-syn-1-0.0 Request processed in 148us (591us total) trans 17179874641 rc 0/0 00000100:00100000:1.0:1769074882.331888:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14537 00000100:00100000:1.0:1769074882.331935:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874637 gen 2 00000004:00080000:1.0:1769074882.331941:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d0e4a80, transno 17179874636 00000004:00080000:1.0:1769074882.331951:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d0e6680, transno 17179874637 00000100:00100000:1.0:1769074882.331957:0:3651:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b67a0f6d80 x1855007625255296/t17179874641(17179874641) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074898 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1769074882.331967:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a0f6d80/2, rc 0, transno 17179874641 00000100:00100000:1.0:1769074882.331972:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625255296:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769074882.332390:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074882.337766:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617047040 00000100:00100000:0.0:1769074882.337785:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617047040 00000100:00100000:0.0:1769074882.337901:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13622 00000100:00100000:0.0:1769074882.337912:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:67463:x1855007617047040:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074882.337939:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074882.337960:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55ba2 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c93c expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074882.338018:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55ba2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c93c expref: 22 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074882.338031:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55ba2 lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c93c expref: 22 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074882.338049:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:0.0:1769074882.338052:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074882.338055:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135e00, rc 0) 00000100:00100000:0.0:1769074882.338124:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67463:x1855007617047040:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 213us (5737us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074882.338132:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13622 00000004:00080000:2.0:1769074882.344694:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f28e00/2, rc 0, transno 21474841311 00000100:00100000:2.0:1769074882.344710:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625255168:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1769074882.344923:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074882.345192:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255424 00000100:00100000:1.0:1769074882.345202:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255424 00000100:00100000:1.0:1769074882.345297:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14538 00000100:00100000:1.0:1769074882.345307:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625255424:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1769074882.345453:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:1.0:1769074882.345456:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb mtime from disk: 1769074881 -> 1769074881 00002000:00010000:1.0:1769074882.345459:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074882.345461:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb ctime from disk: 1769074881 -> 1769074881 00002000:00010000:1.0:1769074882.345464:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000401:0x2cc:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074882.345542:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625255424:12345-0@lo:2:osp-syn-1-0.0 Request processed in 236us (619us total) trans 17179874642 rc 0/0 00000100:00100000:1.0:1769074882.345549:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14538 00000004:00080000:3.0:1769074882.345617:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b67a0f5180/2, rc 0, transno 17179874642 00000100:00100000:3.0:1769074882.345623:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625255424:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769074882.350965:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.351078:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617047296 00000100:00100000:2.0:1769074882.351087:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617047296 00000100:00100000:2.0:1769074882.351139:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4747 00000100:00100000:2.0:1769074882.351150:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67463:x1855007617047296:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074882.351384:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67463:x1855007617047296:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 234us (424us total) trans 8589944137 rc 0/0 00000100:00100000:2.0:1769074882.351392:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4747 00000100:00100000:1.0:1769074882.367922:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.368086:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617047552 00000100:00100000:2.0:1769074882.368098:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617047552 00000100:00100000:2.0:1769074882.368154:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13623 00000100:00100000:2.0:1769074882.368164:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67464:x1855007617047552:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074882.368190:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074882.368212:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55ba9 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c943 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.368304:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368318:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368330:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368342:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368348:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368355:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368429:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368436:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368442:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368449:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368454:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.368459:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368465:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368490:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.368494:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.368498:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368504:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55bb0 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368519:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368525:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368530:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368537:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.368540:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368545:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368551:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effa200) 00010000:00010000:2.0:1769074882.368553:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.368556:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368561:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bb7 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.368592:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55ba9 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c943 expref: 24 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.368601:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.368603:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074882.368606:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55ba9 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c943 expref: 24 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.368616:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074882.368681:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67464:x1855007617047552:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 518us (770us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074882.368689:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13623 00000100:00100000:0.0:1769074882.378210:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074882.378301:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617047680 00000100:00100000:3.0:1769074882.378312:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617047680 00000100:00100000:3.0:1769074882.378367:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13624 00000100:00100000:3.0:1769074882.378378:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67464:x1855007617047680:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074882.378456:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378480:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378493:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378505:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378511:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378516:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378595:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378605:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378611:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074882.378651:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097152 00000004:00200000:3.0:1769074882.378655:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074882.378667:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378680:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378687:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378693:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378699:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378705:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.378708:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378715:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378725:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d700) 00010000:00010000:3.0:1769074882.378729:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.378734:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378739:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55bbe lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378757:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378762:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378767:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378771:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.378774:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378779:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378785:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.378787:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.378789:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378794:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55bc5 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378804:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378851:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378857:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378861:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.378864:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378870:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378876:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074882.378878:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.378881:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.378887:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bcc lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074882.378959:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67464:x1855007617047680:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 583us (759us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074882.378967:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13624 00000100:00100000:1.0:1769074882.429931:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.430108:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617047936 00000100:00100000:2.0:1769074882.430120:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617047936 00000100:00100000:2.0:1769074882.430174:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13625 00000100:00100000:2.0:1769074882.430185:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67469:x1855007617047936:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074882.430206:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074882.430231:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55bd3 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c94a expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.430309:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430323:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430334:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430346:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430352:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430358:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430427:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430435:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430440:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430450:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430455:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.430459:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430480:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430493:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.430496:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.430500:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430505:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55bda lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430518:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430524:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430529:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430534:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.430537:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430543:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430553:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effa200) 00010000:00010000:2.0:1769074882.430556:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.430558:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430563:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55be1 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.430593:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55bd3 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c94a expref: 24 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.430603:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.430605:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074882.430608:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55bd3 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c94a expref: 24 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.430619:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074882.430704:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67469:x1855007617047936:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 521us (783us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074882.430712:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13625 00000100:00100000:1.0:1769074882.438697:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074882.438802:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617048064 00000100:00100000:0.0:1769074882.438853:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617048064 00000100:00100000:0.0:1769074882.438911:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13626 00000100:00100000:0.0:1769074882.438920:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67469:x1855007617048064:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074882.438992:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439009:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439019:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439030:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439036:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439042:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439120:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439128:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439134:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:0.0:1769074882.439177:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097152 00000004:00200000:0.0:1769074882.439181:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074882.439194:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439207:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439213:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439219:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439226:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439232:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074882.439237:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439244:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439256:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b67a317800) 00010000:00010000:0.0:1769074882.439259:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074882.439263:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439269:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135c00/0x7fe2323adfd55be8 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439285:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439291:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439296:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439301:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074882.439303:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439309:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439315:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b67a317600) 00010000:00010000:0.0:1769074882.439317:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074882.439319:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439325:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135200/0x7fe2323adfd55bef lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439335:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439340:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439345:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439349:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074882.439352:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439358:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.439364:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:0.0:1769074882.440084:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074882.440090:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074882.440098:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd55bf6 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074882.440638:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67469:x1855007617048064:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1713us (1945us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074882.440650:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13626 00000100:00100000:2.0:1769074882.471021:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074882.471047:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b425880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625255552:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074882.471143:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074882.471260:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255552 00000100:00100000:3.0:1769074882.471269:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255552 00000100:00100000:3.0:1769074882.471344:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1255 00000100:00100000:3.0:1769074882.471353:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625255552:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074882.471377:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3682287616 00000100:00100000:3.0:1769074882.471428:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625255552:12345-0@lo:13:osp-pre-1-1.0 Request processed in 75us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074882.471436:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1255 00000100:00100000:3.0:1769074882.471501:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b425880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625255552:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1769074882.471982:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074882.496543:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074882.497231:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617048320 00000100:00100000:3.0:1769074882.497242:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617048320 00000100:00100000:3.0:1769074882.497302:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13627 00000100:00100000:3.0:1769074882.497312:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67475:x1855007617048320:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074882.497330:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074882.497354:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bfd lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c951 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.497432:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497446:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497458:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497483:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497490:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497497:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497565:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497573:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497580:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497586:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497591:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.497595:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497602:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497613:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.497616:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.497620:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497625:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c04 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497638:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497643:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497648:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497653:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.497656:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497660:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497666:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19d700) 00010000:00010000:3.0:1769074882.497669:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.497671:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497677:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c0b lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.497706:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bfd lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c951 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.497716:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.497718:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074882.497721:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55bfd lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c951 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.497731:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbc00, rc 0) 00000100:00100000:3.0:1769074882.501892:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67475:x1855007617048320:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4570us (5348us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074882.501905:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13627 00000100:00100000:1.0:1769074882.509232:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074882.516988:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617048448 00000100:00100000:3.0:1769074882.517004:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617048448 00000100:00100000:3.0:1769074882.517087:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13628 00000100:00100000:3.0:1769074882.517098:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67475:x1855007617048448:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074882.517183:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517200:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517210:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517222:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517229:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517235:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517310:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517318:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517324:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074882.517363:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097152 00000004:00200000:3.0:1769074882.517367:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074882.517379:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517393:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517402:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517407:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517414:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517418:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.517422:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517427:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517437:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.517440:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.517443:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517448:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c12 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517464:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517485:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517490:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517494:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.517498:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517503:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517510:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19d700) 00010000:00010000:3.0:1769074882.517512:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.517515:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517521:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c19 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517531:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517537:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517541:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517545:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.517547:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517552:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517558:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074882.517560:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.517562:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.517570:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c20 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074882.517654:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67475:x1855007617048448:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 557us (8430us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074882.517662:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13628 00000100:00100000:1.0:1769074882.571537:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.571625:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617048704 00000100:00100000:2.0:1769074882.571635:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617048704 00000100:00100000:2.0:1769074882.571698:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13629 00000100:00100000:2.0:1769074882.571708:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67484:x1855007617048704:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074882.571728:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074882.571754:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c27 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c958 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.571881:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.571896:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.571908:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.571919:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.571925:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.571932:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572002:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572009:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572015:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572021:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572027:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.572031:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572039:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572050:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.572053:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.572058:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572064:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c2e lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572077:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572084:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572089:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572094:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.572097:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572103:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572110:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effa200) 00010000:00010000:2.0:1769074882.572112:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.572115:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572121:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c35 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.572167:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c27 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c958 expref: 24 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.572178:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.572180:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074882.572184:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c27 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c958 expref: 24 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074882.572195:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074882.572259:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67484:x1855007617048704:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 552us (733us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074882.572267:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13629 00000100:00100000:1.0:1769074882.583004:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074882.583096:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617048832 00000100:00100000:3.0:1769074882.583107:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617048832 00000100:00100000:3.0:1769074882.583173:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13630 00000100:00100000:3.0:1769074882.583183:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67484:x1855007617048832:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074882.583260:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583275:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583287:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583311:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583317:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583322:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583389:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583398:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583406:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074882.583444:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097152 00000004:00200000:3.0:1769074882.583448:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074882.583461:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583484:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583490:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583496:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583501:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583506:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.583510:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583517:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583528:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d700) 00010000:00010000:3.0:1769074882.583532:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.583536:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583541:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c3c lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583559:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583564:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583568:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583573:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.583575:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583581:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583588:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.583590:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.583592:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583598:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c43 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583606:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583611:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583617:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583636:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.583639:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583645:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583652:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:3.0:1769074882.583655:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.583658:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.583664:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c4a lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00000100:00100000:3.0:1769074882.583785:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67484:x1855007617048832:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 602us (789us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074882.583794:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13630 00000100:00100000:1.0:1769074882.609225:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074882.609323:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049088 00000100:00100000:3.0:1769074882.609333:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049088 00000100:00100000:3.0:1769074882.609396:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13631 00000100:00100000:3.0:1769074882.609406:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67486:x1855007617049088:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074882.609425:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074882.609462:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c51 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c95f expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.609552:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609565:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609576:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609588:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609595:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609601:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609680:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609688:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609695:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609702:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609708:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.609712:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609719:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609730:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.609734:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.609738:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609744:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55c58 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609756:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609762:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609769:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609773:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074882.609776:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609782:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609788:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64a19d700) 00010000:00010000:3.0:1769074882.609791:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074882.609793:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609799:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55c5f lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074882.609898:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c51 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c95f expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.609910:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64a19d800) 00010000:00010000:3.0:1769074882.609913:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074882.609917:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55c51 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c95f expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074882.609928:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbc00, rc 0) 00000100:00100000:3.0:1769074882.609990:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67486:x1855007617049088:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 584us (776us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074882.609998:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13631 00000100:00100000:1.0:1769074882.615009:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074882.615110:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049216 00000100:00100000:2.0:1769074882.615119:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049216 00000100:00100000:2.0:1769074882.615179:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13632 00000100:00100000:2.0:1769074882.615189:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67486:x1855007617049216:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074882.615263:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615279:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615290:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615301:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615308:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615315:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615381:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615391:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615397:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:2.0:1769074882.615435:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097152 00000004:00200000:2.0:1769074882.615534:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074882.615548:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615563:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615569:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615574:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615583:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615588:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.615592:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615599:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615611:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa700) 00010000:00010000:2.0:1769074882.615614:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.615712:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615720:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c66 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615739:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615744:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615749:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615753:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.615756:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615762:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615768:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effa200) 00010000:00010000:2.0:1769074882.615770:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.615773:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615779:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c6d lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615790:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615795:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615800:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615940:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074882.615946:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615952:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615961:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074882.615963:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074882.615966:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074882.615971:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c74 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:2.0:1769074882.616048:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67486:x1855007617049216:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 858us (1048us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074882.616057:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13632 00010000:00010000:2.0:1769074882.973110:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074884.000976:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074884.511019:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625255680:0@lo:400:kworker.0 00000100:00100000:3.0:1769074884.511096:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074884.511212:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255680 00000100:00100000:1.0:1769074884.511220:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255680 00000100:00100000:1.0:1769074884.511271:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1769074884.511281:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625255680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.512864:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625256064:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.512968:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074884.513078:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625255680:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.517655:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256064 00000100:00100000:2.0:1769074884.517682:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256064 00000100:00100000:2.0:1769074884.517780:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1573 00000100:00100000:2.0:1769074884.517793:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625256064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.517931:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625256064:12345-0@lo:400:kworker.0 Request processed in 141us (4964us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074884.517941:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1573 00000100:00080000:2.0:1769074884.517974:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074884.518020:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625256064:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.519140:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625255936:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.519234:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074884.519267:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625256448:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.521640:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625256576:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.521735:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074884.521869:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625256960:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.521929:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074884.521958:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1769074884.521965:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625255808:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.522036:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074884.522058:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625256320:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.522079:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074884.523422:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625255680:12345-0@lo:400:kworker.0 Request processed in 12134us (12316us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074884.523446:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1769074884.525756:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625256832:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.528208:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074884.528286:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255808 00000100:00100000:0.0:1769074884.528450:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255808 00000100:00100000:0.0:1769074884.528515:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256832 00000100:00100000:0.0:1769074884.528520:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256832 00000100:00100000:0.0:1769074884.528541:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2749 00000100:00100000:0.0:1769074884.528550:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625255808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.528887:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625255808:12345-0@lo:400:kworker.0 Request processed in 337us (6849us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074884.528895:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2749 00000100:00100000:0.0:1769074884.528914:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2750 00000100:00100000:0.0:1769074884.528920:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625256832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.528951:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625256832:12345-0@lo:400:kworker.0 Request processed in 31us (753us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074884.528957:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2750 00000100:00100000:1.0:1769074884.530653:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625255936 00000100:00100000:1.0:1769074884.530693:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625255936 00000100:00100000:1.0:1769074884.530951:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256448 00000100:00100000:1.0:1769074884.530960:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256448 00000100:00100000:1.0:1769074884.532528:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256320 00000100:00100000:1.0:1769074884.532543:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256320 00000100:00100000:1.0:1769074884.532581:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2495 00000100:00100000:1.0:1769074884.532594:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625255936:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074884.532720:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625255936:12345-0@lo:400:kworker.0 Request processed in 130us (13485us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074884.533166:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2495 00000100:00100000:1.0:1769074884.533201:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2496 00000100:00100000:1.0:1769074884.533210:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625256448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.533359:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2497 00000100:00100000:2.0:1769074884.533372:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625256320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074884.533498:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625256448:12345-0@lo:400:kworker.0 Request processed in 286us (11770us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074884.533505:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2496 00000100:00100000:2.0:1769074884.533638:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625256320:12345-0@lo:400:kworker.0 Request processed in 270us (11557us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074884.533645:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2497 00000100:00100000:2.0:1769074884.533677:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074884.534228:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625257088:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.534279:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074884.534316:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074884.534323:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625256192:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.534346:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074884.534368:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625256704:0@lo:400:kworker.0 00000100:00100000:1.0:1769074884.534526:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256576 00000100:00100000:1.0:1769074884.534535:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256576 00000100:00100000:1.0:1769074884.534580:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2498 00000100:00100000:1.0:1769074884.534589:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625256576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074884.534632:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625256576:12345-0@lo:400:kworker.0 Request processed in 45us (955us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074884.534639:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2498 00000100:00100000:2.0:1769074884.535990:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074884.536027:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625257216:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.536062:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074884.536121:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625256576:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.536176:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256960 00000100:00100000:2.0:1769074884.536184:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256960 00000100:00100000:2.0:1769074884.536229:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257088 00000100:00100000:2.0:1769074884.536232:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257088 00000100:00100000:2.0:1769074884.536257:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256192 00000100:00100000:2.0:1769074884.536260:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256192 00000100:00100000:2.0:1769074884.536286:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1574 00000100:00100000:2.0:1769074884.536295:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625256960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.536345:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625256960:12345-0@lo:400:kworker.0 Request processed in 51us (14415us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074884.536352:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1574 00000100:00100000:2.0:1769074884.536370:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1575 00000100:00100000:2.0:1769074884.536376:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625257088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536472:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625256704 00000100:00100000:0.0:1769074884.536488:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625256704 00000100:00100000:0.0:1769074884.536554:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257216 00000100:00100000:0.0:1769074884.536558:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257216 00000100:00100000:0.0:1769074884.536603:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2499 00000100:00100000:0.0:1769074884.536613:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625256704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536709:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625256704:12345-0@lo:400:kworker.0 Request processed in 98us (726us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074884.536716:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2499 00000100:00100000:0.0:1769074884.536733:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2500 00000100:00100000:0.0:1769074884.536740:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625257216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536769:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625257216:12345-0@lo:400:kworker.0 Request processed in 30us (707us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074884.536774:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2500 00000100:00100000:0.0:1769074884.536843:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625255808:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536872:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625256320:0@lo:400:kworker.0 00000100:00100000:1.0:1769074884.536873:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1576 00000100:00100000:1.0:1769074884.536895:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625256192:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536903:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625256832:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536921:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625255936:0@lo:400:kworker.0 00000100:00100000:0.0:1769074884.536950:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625256448:0@lo:400:kworker.0 00000100:00100000:1.0:1769074884.536955:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625256192:12345-0@lo:400:kworker.0 Request processed in 66us (2604us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074884.536968:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1576 00000100:00100000:0.0:1769074884.536978:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625256960:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.537568:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625257088:12345-0@lo:400:kworker.0 Request processed in 1190us (3286us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074884.537576:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1575 00000100:00080000:2.0:1769074884.537614:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074884.537619:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074884.537623:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074884.537626:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074884.537678:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625256192:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.537693:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625256704:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.537706:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625257216:0@lo:400:kworker.0 00000100:00100000:2.0:1769074884.537719:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625257088:0@lo:400:kworker.0 00010000:00010000:1.0:1769074885.055733:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074885.593721:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074885.593788:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074885.593863:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074885.593887:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074885.593918:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074885.593956:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049344 00000100:00100000:1.0:1769074885.593970:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049344 00000100:00100000:1.0:1769074885.594035:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 387 00000100:00100000:1.0:1769074885.594046:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617049344:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074885.594139:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617049344:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 95us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074885.594148:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 387 00000100:00100000:1.0:1769074885.594191:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049472 00000100:00100000:1.0:1769074885.594195:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049472 00000100:00100000:1.0:1769074885.594238:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049600 00000100:00100000:1.0:1769074885.594241:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049600 00000100:00100000:1.0:1769074885.594265:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2501 00000100:00100000:1.0:1769074885.594272:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007617049472:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074885.594646:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2502 00000100:00100000:0.0:1769074885.594657:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617049600:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074885.595305:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049728 00000100:00100000:1.0:1769074885.595462:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007617049472:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 1187us (1753us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074885.595472:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2501 00000100:00100000:2.0:1769074885.595676:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049728 00000100:00100000:2.0:1769074885.595729:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049856 00000100:00100000:2.0:1769074885.595733:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049856 00000100:00100000:2.0:1769074885.595757:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1577 00000100:00100000:2.0:1769074885.595764:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617049728:12345-192.168.204.60@tcp:400:kworker.0 00000100:00080000:3.0:1769074885.598886:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074885.598899:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074885.598903:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074885.598907:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074885.598950:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1578 00000100:00100000:3.0:1769074885.598974:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617049856:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074885.599922:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617049600:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 5259us (6025us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074885.599939:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2502 00000100:00100000:2.0:1769074885.602184:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617049728:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 6403us (8377us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074885.602205:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1577 00000100:00100000:3.0:1769074885.605652:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617049856:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 6673us (11715us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074885.605672:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1578 00010000:00010000:2.0:1769074886.045219:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074886.558987:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625257344:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074886.559055:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074886.559092:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074886.559098:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625257472:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074886.559121:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074886.559155:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257344 00000100:00100000:2.0:1769074886.559162:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257344 00000100:00100000:2.0:1769074886.559207:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257472 00000100:00100000:2.0:1769074886.559210:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257472 00000100:00100000:2.0:1769074886.559891:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2751 00000100:00100000:2.0:1769074886.559900:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625257344:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074886.559935:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074886.559997:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625257344:12345-0@lo:41:osp-pre-1-0.0 Request processed in 98us (941us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074886.560005:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2751 00000100:00100000:2.0:1769074886.560024:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2752 00000100:00100000:2.0:1769074886.560030:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625257472:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074886.560047:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:1.0:1769074886.560051:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625257344:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074886.560075:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625257472:12345-0@lo:41:osp-pre-0-1.0 Request processed in 45us (954us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074886.560077:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625257472:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074886.560083:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2752 00010000:00010000:2.0:1769074887.069709:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074887.070445:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625257600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074887.070508:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074887.070567:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074887.070574:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa21f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625257728:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074887.070596:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074887.070622:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074887.070630:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625257856:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074887.070639:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257600 00000100:00100000:0.0:1769074887.070647:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257600 00000100:00100000:2.0:1769074887.070654:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074887.070683:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257728 00000100:00100000:2.0:1769074887.070687:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257728 00000100:00100000:0.0:1769074887.070691:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257856 00000100:00100000:0.0:1769074887.070694:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257856 00000100:00100000:0.0:1769074887.070708:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1256 00000100:00100000:0.0:1769074887.070773:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625257600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769074887.070803:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:0.0:1769074887.070884:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625257600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 170us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074887.070954:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625257600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074887.076871:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1257 00000100:00100000:2.0:1769074887.076933:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1258 00000100:00100000:2.0:1769074887.076945:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625257856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074887.076986:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:2.0:1769074887.077077:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625257856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 136us (6421us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074887.077085:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1258 00000100:00100000:1.0:1769074887.077128:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625257856:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074887.077128:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625257728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074887.077145:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3913859072 avail 3682336768 00000100:00100000:2.0:1769074887.077185:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625257728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 302us (6588us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074887.077191:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1257 00000100:00100000:1.0:1769074887.077522:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa21f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625257728:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1769074887.077663:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074887.077669:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074887.079402:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1256 00000001:00080000:2.0:1769074887.083268:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841311 is committed 00000001:00080000:2.0:1769074887.083282:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841310 is committed 00000100:00100000:3.0:1769074887.581430:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1769074887.581462:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67fa21f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625257984:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074887.581544:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074887.581693:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625257984 00000100:00100000:0.0:1769074887.581702:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625257984 00000100:00100000:0.0:1769074887.581775:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1259 00000100:00100000:0.0:1769074887.581785:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625257984:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074887.581839:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3913859072 avail 3682336768 00000100:00100000:0.0:1769074887.581886:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625257984:12345-0@lo:13:osp-pre-1-1.0 Request processed in 103us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074887.581894:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1259 00000100:00080000:3.0:1769074887.582128:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074887.582164:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67fa21f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625257984:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1769074888.093077:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1769074888.100124:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944137 is committed 00000001:00080000:2.0:1769074888.100624:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874642 is committed 00000001:00080000:2.0:1769074888.100636:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874640 is committed 00000100:00100000:1.0:1769074888.963717:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074888.963845:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617049984 00000100:00100000:2.0:1769074888.963855:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617049984 00000100:00100000:2.0:1769074888.963923:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13633 00000100:00100000:2.0:1769074888.963933:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67905:x1855007617049984:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:2.0:1769074888.963953:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074888.963976:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55c7b lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c966 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074888.964046:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964060:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964074:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964088:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964094:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964100:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964172:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964186:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964194:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:2.0:1769074888.964233:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20a:0x0]: returning size 2097152 00000004:00200000:2.0:1769074888.964239:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074888.964271:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964284:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964294:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964300:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964306:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964312:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964318:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074888.964322:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964329:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964341:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074888.964345:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074888.964350:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964356:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55c82 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964383:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964389:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964394:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964399:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074888.964402:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964408:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964415:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effae00) 00010000:00010000:2.0:1769074888.964418:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074888.964421:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964426:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55c89 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:2.0:1769074888.964457:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55c7b lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c966 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074888.964478:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c966 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074888.964486:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074888.964488:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074888.964491:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074888.964562:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67905:x1855007617049984:12345-192.168.204.60@tcp:101:rm.0 Request processed in 630us (859us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074888.964570:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13633 00000100:00080000:1.0:1769074888.964900:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074888.971440:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074888.975889:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617050240 00000100:00100000:2.0:1769074888.975905:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617050240 00000100:00100000:2.0:1769074888.975988:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13634 00000100:00100000:2.0:1769074888.976007:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:67905:x1855007617050240:12345-192.168.204.60@tcp:36:rm.0 00010000:00010000:2.0:1769074888.976060:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1769074888.976088:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd55ba2 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c93c expref: 23 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074888.976109:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55c90 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c966 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074888.976122:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549242000/0x7fe2323adfd55a13 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c8c5 expref: 21 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074888.976133:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b681697600) 00010000:00010000:2.0:1769074888.976145:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074888.976169:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 3/0,1 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976182:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976191:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976204:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976210:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976214:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976302:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976310:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.976317:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000040:00080000:2.0:1769074888.976574:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].2474, 64 off211352 00000004:00080000:2.0:1769074888.976584:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1876818777343ns, next at 1877818774942ns, rc = 0 00000040:00080000:2.0:1769074888.976606:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2480, 64 off212120 00000004:00080000:2.0:1769074888.976613:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1876818805928ns, next at 1877818804795ns, rc = 0 00000040:00080000:2.0:1769074888.976622:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].2481, 64 off212184 00010000:00010000:2.0:1769074888.977611:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 3/0,1 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977629:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977639:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977648:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977653:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 4/0,1 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977658:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977771:0:6512:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1769074888.977797:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977817:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977824:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977829:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074888.977833:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977839:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977851:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b64effaa00) 00010000:00010000:2.0:1769074888.977855:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1769074888.977859:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 2/0,0 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.977868:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89400/0x7fe2323adfd55ca5 lrc: 0/0,0 mode: --/EX res: [0x200000405:0x20a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074888.977918:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074888.977934:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074888.977946:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074888.977951:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074888.977962:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 3/0,0 mode: PW/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074888.977981:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20a:0x0].0x0 (ffff89b64effaa00) 00010000:00010000:3.0:1769074888.977983:0:6501:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1769074888.977987:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 1/0,0 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074888.977991:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd55cac lrc: 0/0,0 mode: --/PW res: [0x200000405:0x20a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074888.978029:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074888.978032:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074888.978034:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074888.978051:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978060:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978065:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 3/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978071:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978076:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074888.978080:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978086:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 5/0,0 mode: CW/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978095:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074888.978098:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074888.978101:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 3/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978107:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55c97 lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000004:00010000:2.0:1769074888.978125:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000677c87e4 reply state 0000000061f94b97 transno 8589944138 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978135:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074888.978142:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effac00) 00010000:00010000:2.0:1769074888.978144:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074888.978148:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074888.978252:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa23800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:67905:x1855007617050240:12345-192.168.204.60@tcp:36:rm.0 Request processed in 2250us (6816us total) trans 8589944138 rc 0/0 00000100:00100000:2.0:1769074888.978261:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13634 00010000:00010000:2.0:1769074889.117049:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074889.629288:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625258112:0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.629432:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074889.629490:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625258624:0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.629522:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074889.629558:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625259136:0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.629592:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074889.629618:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625259648:0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.629645:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074889.629672:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1769074889.629678:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625258240:0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.629711:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074889.629727:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259136 00000100:00100000:3.0:1769074889.629733:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625258752:0@lo:400:kworker.0 00000100:00100000:0.0:1769074889.629740:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259136 00000100:00100000:3.0:1769074889.629760:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074889.629783:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625259264:0@lo:400:kworker.0 00000100:00100000:0.0:1769074889.629848:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259648 00000100:00100000:3.0:1769074889.629849:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074889.629854:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259648 00000100:00100000:0.0:1769074889.629882:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258752 00000100:00100000:0.0:1769074889.629885:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258752 00000100:00100000:3.0:1769074889.629890:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258624 00000100:00100000:3.0:1769074889.629896:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258624 00000100:00100000:3.0:1769074889.629935:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1579 00000100:00100000:3.0:1769074889.629945:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625258624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.630011:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625258624:12345-0@lo:400:kworker.0 Request processed in 68us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074889.630021:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1579 00000100:00100000:3.0:1769074889.630060:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2503 00000100:00100000:3.0:1769074889.630068:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625259136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.630103:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625259136:12345-0@lo:400:kworker.0 Request processed in 36us (509us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074889.630110:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2503 00000100:00100000:1.0:1769074889.630114:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258112 00000100:00100000:1.0:1769074889.630120:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258112 00000100:00100000:3.0:1769074889.630130:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2504 00000100:00100000:3.0:1769074889.630140:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625259648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630164:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1769074889.630175:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625259648:12345-0@lo:400:kworker.0 Request processed in 36us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074889.630177:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625258112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074889.630189:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2504 00000100:00100000:3.0:1769074889.630206:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2505 00000100:00100000:3.0:1769074889.630214:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625258752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630235:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625258112:12345-0@lo:400:kworker.0 Request processed in 60us (802us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074889.630244:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1769074889.630252:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625258752:12345-0@lo:400:kworker.0 Request processed in 38us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074889.630258:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2505 00000100:00100000:2.0:1769074889.630305:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258240 00000100:00100000:2.0:1769074889.630310:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258240 00000100:00100000:2.0:1769074889.630344:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259264 00000100:00100000:2.0:1769074889.630356:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259264 00000100:00100000:2.0:1769074889.630381:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2753 00000100:00100000:2.0:1769074889.630389:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64efea850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625258240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.630433:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64efea850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625258240:12345-0@lo:400:kworker.0 Request processed in 45us (722us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074889.630440:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2753 00000100:00080000:1.0:1769074889.630519:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074889.630525:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074889.630528:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074889.630567:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625258368:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630600:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074889.630622:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625258880:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630645:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074889.630664:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625259392:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630685:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074889.630708:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1769074889.630715:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625258496:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630738:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074889.630757:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625259008:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630778:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074889.630790:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625259520:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.630848:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074889.630893:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258368 00000100:00100000:1.0:1769074889.630897:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258368 00000100:00100000:2.0:1769074889.631181:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2754 00000100:00100000:2.0:1769074889.631191:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625259264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.631233:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625259264:12345-0@lo:400:kworker.0 Request processed in 44us (1426us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074889.631239:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2754 00000100:00100000:2.0:1769074889.631271:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259392 00000100:00100000:2.0:1769074889.631275:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259392 00000100:00100000:2.0:1769074889.631309:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258496 00000100:00100000:2.0:1769074889.631312:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258496 00000100:00100000:2.0:1769074889.631332:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259520 00000100:00100000:2.0:1769074889.631335:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259520 00000100:00100000:2.0:1769074889.631369:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1580 00000100:00100000:2.0:1769074889.631377:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625259392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.631415:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625259392:12345-0@lo:400:kworker.0 Request processed in 39us (728us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074889.631422:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1580 00000100:00100000:2.0:1769074889.631434:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1581 00000100:00100000:2.0:1769074889.631440:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625258496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.631465:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625258496:12345-0@lo:400:kworker.0 Request processed in 25us (726us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074889.631471:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1581 00000100:00100000:2.0:1769074889.631480:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1582 00000100:00100000:2.0:1769074889.631485:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625259520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.631508:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625259520:12345-0@lo:400:kworker.0 Request processed in 24us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074889.631514:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1582 00000100:00080000:2.0:1769074889.631560:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074889.631563:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074889.631598:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625258496:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.631610:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874642 gen 2 00000004:00080000:2.0:1769074889.631616:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a0f6d80, transno 17179874641 00000004:00080000:2.0:1769074889.631631:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b67a0f5180, transno 17179874642 00000100:00100000:2.0:1769074889.631636:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625259520:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.631646:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841311 gen 2 00000004:00080000:2.0:1769074889.631649:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54b426a00, transno 21474841309 00000004:00080000:2.0:1769074889.631657:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f28e00, transno 21474841311 00000100:00100000:2.0:1769074889.631662:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625259392:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.632027:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625258880 00000100:00100000:1.0:1769074889.632033:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625258880 00000100:00100000:1.0:1769074889.632056:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259008 00000100:00100000:1.0:1769074889.632060:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259008 00000100:00100000:1.0:1769074889.632075:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2506 00000100:00100000:1.0:1769074889.632082:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625258368:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.632135:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625258368:12345-0@lo:400:kworker.0 Request processed in 54us (1535us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074889.632142:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2506 00000100:00100000:1.0:1769074889.632159:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2507 00000100:00100000:1.0:1769074889.632164:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625258880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.632195:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa20700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625258880:12345-0@lo:400:kworker.0 Request processed in 30us (1548us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074889.632201:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2507 00000100:00100000:1.0:1769074889.632212:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2508 00000100:00100000:1.0:1769074889.632218:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67fa21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625259008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.632243:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67fa21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625259008:12345-0@lo:400:kworker.0 Request processed in 25us (1464us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074889.632249:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2508 00000100:00080000:1.0:1769074889.632264:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074889.632268:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074889.632291:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625259008:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.632302:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625258368:0@lo:400:kworker.0 00000100:00100000:1.0:1769074889.632313:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625258880:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633240:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625258240:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633271:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625258752:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633287:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625259264:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633302:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625259136:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633315:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625259648:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633330:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625258112:0@lo:400:kworker.0 00000100:00100000:2.0:1769074889.633343:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625258624:0@lo:400:kworker.0 00010000:00010000:1.0:1769074890.141295:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074890.694404:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074890.694449:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074890.694478:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074890.694498:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074890.694520:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074890.694537:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617050624 00000100:00100000:2.0:1769074890.694545:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617050496 00000100:00100000:0.0:1769074890.694549:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617050624 00000100:00100000:2.0:1769074890.694551:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617050496 00000100:00100000:2.0:1769074890.694617:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2509 00000100:00100000:2.0:1769074890.694627:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007617050496:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074890.694703:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:2408:x1855007617050496:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 77us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074890.694709:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2509 00000100:00100000:2.0:1769074890.694731:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617050880 00000100:00100000:2.0:1769074890.694735:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617050880 00000100:00100000:3.0:1769074890.694794:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617050368 00000100:00100000:3.0:1769074890.694798:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617050368 00000100:00100000:3.0:1769074890.694839:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 389 00000100:00100000:3.0:1769074890.694847:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6723f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617050368:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074890.694876:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6723f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617050368:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 31us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074890.694882:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 389 00000100:00100000:1.0:1769074890.696109:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617050752 00000100:00100000:1.0:1769074890.696133:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617050752 00000100:00100000:0.0:1769074890.698174:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2510 00000100:00100000:0.0:1769074890.698211:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617050624:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074890.700411:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617050624:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 2213us (5947us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074890.700428:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2510 00000100:00080000:3.0:1769074890.702757:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074890.702770:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074890.702774:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074890.702777:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074890.711564:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1583 00000100:00100000:1.0:1769074890.711581:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617050752:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074890.711673:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617050752:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 96us (17151us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074890.711681:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1583 00000100:00100000:2.0:1769074890.711718:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1584 00000100:00100000:2.0:1769074890.711727:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617050880:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074890.711757:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617050880:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (17279us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074890.711764:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1584 00000001:02000400:3.0:1769074891.115622:0:38428:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 204d: FLR write/stale/resync sel test with repeated comp ========================================================== 04:41:28 (1769074888) 00010000:00010000:3.0:1769074891.166667:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074891.679589:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074891.679614:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625259776:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074891.679686:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074891.679923:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074891.679934:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625259904:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074891.679961:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074891.680073:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259776 00000100:00100000:2.0:1769074891.680083:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259776 00000100:00100000:2.0:1769074891.680137:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625259904 00000100:00100000:2.0:1769074891.680141:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625259904 00000100:00100000:2.0:1769074891.680170:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2755 00000100:00100000:2.0:1769074891.680180:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625259776:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074891.680213:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074891.680261:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625259776:12345-0@lo:41:osp-pre-1-0.0 Request processed in 82us (574us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074891.680268:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2755 00000100:00100000:2.0:1769074891.680287:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2756 00000100:00100000:2.0:1769074891.680293:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625259904:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074891.680310:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:2.0:1769074891.680353:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625259904:12345-0@lo:41:osp-pre-0-1.0 Request processed in 61us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074891.680359:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2756 00000100:00100000:3.0:1769074891.680479:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625259904:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074891.680509:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625259776:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1769074892.190750:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074892.192825:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625260032:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074892.192896:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074892.192976:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260032 00000100:00100000:1.0:1769074892.192984:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260032 00000100:00100000:1.0:1769074892.193037:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1260 00000100:00100000:1.0:1769074892.193047:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625260032:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074892.193076:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074892.193119:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625260032:12345-0@lo:13:osp-pre-0-1.0 Request processed in 75us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074892.193127:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1260 00000100:00080000:1.0:1769074892.193150:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074892.193188:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625260032:0@lo:13:osp-pre-0-1.0 00000004:00080000:1.0:1769074892.193300:0:32395:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1880035492317ns, next at 1881035490390ns, rc = 0 00000100:00100000:1.0:1769074892.193337:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625260160:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074892.193364:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074892.193394:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260160 00000100:00100000:1.0:1769074892.193398:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260160 00000100:00100000:1.0:1769074892.193425:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1261 00000100:00100000:1.0:1769074892.193430:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625260160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074892.193444:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3912896512 avail 3681374208 00000100:00100000:1.0:1769074892.193472:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625260160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 42us (107us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074892.193548:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1261 00000100:00080000:1.0:1769074892.193564:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074892.193582:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625260160:0@lo:13:osp-pre-0-0.0 00000004:00080000:3.0:1769074892.193718:0:32397:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1880035910362ns, next at 1881035908970ns, rc = 0 00000100:00100000:3.0:1769074892.193735:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625260288:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074892.193764:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074892.194908:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260288 00000100:00100000:1.0:1769074892.194916:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260288 00000100:00100000:1.0:1769074892.194973:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1262 00000100:00100000:1.0:1769074892.194982:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625260288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074892.195005:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3913859072 avail 3682336768 00000100:00100000:1.0:1769074892.195056:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625260288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (1290us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074892.195063:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1262 00000100:00080000:3.0:1769074892.195141:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074892.195174:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625260288:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074892.708544:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625260416:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074892.708625:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074892.708835:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260416 00000100:00100000:0.0:1769074892.708845:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260416 00000100:00100000:0.0:1769074892.708917:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1263 00000100:00100000:0.0:1769074892.708927:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625260416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074892.708950:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3913859072 avail 3682336768 00000100:00100000:0.0:1769074892.708996:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625260416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 71us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074892.709003:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1263 00000100:00080000:1.0:1769074892.709270:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074892.709317:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625260416:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1769074893.213066:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1769074893.697636:0:38524:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1881539826515ns, next at 1881035490390ns, rc = 0 00080000:00080000:2.0:1769074893.697658:0:38524:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:2.0:1769074893.705108:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1974 committed 00000004:00080000:2.0:1769074893.705145:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1977 committed 00000004:00080000:2.0:1769074893.705163:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1973 committed 00000001:00080000:2.0:1769074893.705207:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944138 is committed 00000004:00080000:2.0:1769074893.705215:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1976 committed 00000004:00080000:2.0:1769074893.705219:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1972 committed 00000100:00100000:1.0:1769074893.705301:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074893.705327:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625260544:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1769074893.705399:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074893.705492:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260544 00000100:00100000:1.0:1769074893.705499:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260544 00000100:00100000:1.0:1769074893.705552:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14539 00000100:00100000:1.0:1769074893.705562:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625260544:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1769074893.705600:0:36516:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:713 count 1 00010000:00010000:1.0:1769074893.705629:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.705650:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.705658:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074893.705660:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074893.705664:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.705693:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.705708:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 8 pid: 28955 timeout: 1981 lvb_type: 0 00000100:00100000:1.0:1769074893.705719:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625260672:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074893.705785:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1769074893.706313:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074893.706328:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074893.706336:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074893.706342:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074893.706347:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074893.706355:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074893.706371:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0x9:0x0].0x769e758a (ffff89b64effac00) 00010000:00010000:0.0:1769074893.706375:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074893.706380:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074893.706386:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55c9e lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0x9:0x0].0x769e758a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00000100:00100000:3.0:1769074893.710056:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625260928:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1769074893.710144:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074893.710202:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074893.710209:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625260800:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1769074893.710236:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074893.710286:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260928 00000100:00100000:2.0:1769074893.710305:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260928 00000100:00100000:2.0:1769074893.710385:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625260800 00000100:00100000:2.0:1769074893.710389:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625260800 00000100:00100000:2.0:1769074893.710440:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14540 00000100:00100000:2.0:1769074893.710451:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625260928:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1769074893.710495:0:8398:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:716 count 1 00010000:00010000:2.0:1769074893.710532:0:8398:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.710564:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.710574:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074893.710577:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769074893.710581:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.710680:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 8 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.710698:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 8 pid: 28955 timeout: 1981 lvb_type: 0 00000100:00100000:2.0:1769074893.710711:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625261056:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074893.710775:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b462840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074893.711823:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14541 00000100:00100000:0.0:1769074893.711848:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3650:x1855007625260800:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1769074893.711885:0:28955:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:715 count 1 00010000:00010000:0.0:1769074893.711914:0:28955:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.711933:0:28955:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.711946:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074893.711948:0:28955:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:0.0:1769074893.711952:0:28955:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.711996:0:28955:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 8 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.712007:0:28955:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 8 pid: 8396 timeout: 1981 lvb_type: 0 00000100:00100000:0.0:1769074893.712019:0:28955:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625261184:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074893.712085:0:28955:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b6491f5000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074893.726952:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625260672:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074893.727296:0:36516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074893.733294:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625261056:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074893.733326:0:8398:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074893.733902:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074893.733934:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074893.733983:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074893.734029:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051136 00000100:00100000:1.0:1769074893.734030:0:36300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051264 00000100:00100000:1.0:1769074893.734041:0:36300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051264 00000100:00100000:3.0:1769074893.734047:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051136 00010000:00010000:3.0:1769074893.734111:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 10 pid: 28955 timeout: 1981 lvb_type: 0 00000100:00100000:2.0:1769074893.734134:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051008 00000100:00100000:2.0:1769074893.734143:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051008 00000100:00100000:3.0:1769074893.734168:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7229 00000100:00100000:3.0:1769074893.734181:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617051136:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074893.734202:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074893.734209:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 10 pid: 28955 timeout: 1981 lvb_type: 0 00010000:00010000:3.0:1769074893.734224:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 10 pid: 28955 timeout: 1981 lvb_type: 0 00010000:00010000:3.0:1769074893.734255:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55b86 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c92e expref: 10 pid: 28955 timeout: 1981 lvb_type: 0 00010000:00010000:3.0:1769074893.734307:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074893.734310:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074893.734316:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074893.734329:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074893.734366:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074893.735123:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617051136:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 938us (1218us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074893.735135:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7229 00000100:00100000:0.0:1769074893.736471:0:28955:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:28955:1855007625261184:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074893.736505:0:28955:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.737711:0:36300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 9 pid: 8396 timeout: 1981 lvb_type: 0 00000100:00100000:1.0:1769074893.737773:0:36300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7230 00000100:00100000:1.0:1769074893.737791:0:36300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2406:x1855007617051264:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074893.737825:0:36300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074893.737831:0:36300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 9 pid: 8396 timeout: 1981 lvb_type: 0 00010000:00010000:1.0:1769074893.737843:0:36300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 9 pid: 8396 timeout: 1981 lvb_type: 0 00010000:00010000:1.0:1769074893.737868:0:36300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55b7f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c927 expref: 9 pid: 8396 timeout: 1981 lvb_type: 0 00010000:00010000:1.0:1769074893.737885:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074893.737887:0:36300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074893.737892:0:36300:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.737901:0:36300:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.737932:0:36300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769074893.738406:0:36300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617051264:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 620us (4467us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074893.738420:0:36300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7230 00000100:00080000:1.0:1769074893.738465:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074893.738470:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1769074893.738501:0:28955:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.738511:0:28955:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.738521:0:28955:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.738529:0:28955:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074893.739209:0:28955:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1769074893.740214:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1769074893.740225:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.740238:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074893.740243:0:6501:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.740258:0:6501:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.740290:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.740299:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549135400/0x7fe2323adfd55cc1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28955 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.740327:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074893.740329:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074893.740331:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074893.741120:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 9 pid: 28955 timeout: 1981 lvb_type: 0 00000100:00100000:2.0:1769074893.741178:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7231 00000100:00100000:2.0:1769074893.741196:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617051008:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074893.741214:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074893.741220:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 9 pid: 28955 timeout: 1981 lvb_type: 0 00010000:00010000:2.0:1769074893.741240:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 9 pid: 28955 timeout: 1981 lvb_type: 0 00010000:00010000:2.0:1769074893.741259:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b89200/0x7fe2323adfd55b94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c935 expref: 9 pid: 28955 timeout: 1981 lvb_type: 0 00010000:00010000:2.0:1769074893.741272:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074893.741275:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074893.741290:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.741301:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.741333:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074893.741376:0:8398:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074893.741398:0:8398:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074893.741407:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617051008:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 215us (7419us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074893.741422:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7231 00010000:00010000:3.0:1769074893.741433:0:8398:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074893.741443:0:8398:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074893.741472:0:8398:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1769074893.742252:0:36516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.742266:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.742286:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.742294:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074893.742319:0:36516:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1769074893.742557:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1769074893.742564:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742572:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074893.742575:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742585:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742609:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742617:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55cba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742635:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074893.742638:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742644:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242400 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074893.742647:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742661:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742669:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742675:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549242400/0x7fe2323adfd55cb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074893.742686:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074893.742687:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074893.742689:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074893.753604:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+8:3650:x1855007625260800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 41757us (43355us total) trans 17179874643 rc 0/0 00000100:00100000:1.0:1769074893.753622:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14541 00000004:00080000:2.0:1769074893.754301:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5476f9880/2, rc 0, transno 17179874643 00000100:00100000:2.0:1769074893.754313:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625260800:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1769074893.754635:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074893.757016:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625260928:12345-0@lo:6:osp-syn-1-0.0 Request processed in 46549us (46854us total) trans 17179874644 rc 0/0 00000100:00100000:3.0:1769074893.757036:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14540 00000004:00080000:2.0:1769074893.757122:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b5476fbb80/2, rc 0, transno 17179874644 00000100:00100000:2.0:1769074893.757138:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625260928:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1769074893.758585:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+6:3651:x1855007625260544:12345-0@lo:6:osp-syn-0-0.0 Request processed in 53016us (53178us total) trans 21474841312 rc 0/0 00000100:00100000:0.0:1769074893.758609:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14539 00000004:00080000:1.0:1769074893.766156:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b547599880/2, rc 0, transno 21474841312 00000100:00100000:1.0:1769074893.766228:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625260544:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1769074893.771224:0:38524:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1881613413876ns, next at 1833854042001ns, rc = 0 00080000:00080000:2.0:1769074893.771246:0:38524:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:2.0:1769074893.771345:0:38524:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1881613537232ns, next at 1881035908970ns, rc = 0 00080000:00080000:2.0:1769074893.771351:0:38524:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:2.0:1769074893.771429:0:38524:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1881613621372ns, next at 1835465451293ns, rc = 0 00080000:00080000:2.0:1769074893.771437:0:38524:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:3.0:1769074893.774602:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1978 committed 00000004:00080000:0.0:1769074893.778991:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 69 committed 00000004:00080000:0.0:1769074893.779026:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 65 committed 00010000:00010000:1.0:1769074894.237839:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074894.756760:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625261568:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.757765:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625261312:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.757823:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074894.757873:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625261824:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.757902:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074894.757924:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625262336:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.757966:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625261312 00000100:00100000:1.0:1769074894.757977:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625261312 00000100:00100000:1.0:1769074894.758052:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:1.0:1769074894.758062:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b09c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625261312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.758137:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b09c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625261312:12345-0@lo:400:kworker.0 Request processed in 75us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074894.758145:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:1.0:1769074894.758187:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625261824 00000100:00100000:1.0:1769074894.758191:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625261824 00000100:00100000:1.0:1769074894.758233:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1585 00000100:00100000:1.0:1769074894.758240:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625261824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.758287:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625261824:12345-0@lo:400:kworker.0 Request processed in 47us (383us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074894.758293:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1585 00000100:00080000:1.0:1769074894.758312:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074894.759565:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625261440:0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.759815:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074894.759875:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625262080:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760002:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074894.760032:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625262848:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760070:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074894.760245:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625261312:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760274:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625261824:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760322:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625261568 00000100:00100000:1.0:1769074894.760329:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625261568 00000100:00100000:1.0:1769074894.760378:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262336 00000100:00100000:1.0:1769074894.760381:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262336 00000100:00100000:1.0:1769074894.760409:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262848 00000100:00100000:1.0:1769074894.760413:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262848 00000100:00100000:1.0:1769074894.760430:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2511 00000100:00100000:1.0:1769074894.760437:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625261568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760495:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625261568:12345-0@lo:400:kworker.0 Request processed in 57us (3608us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074894.760502:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2511 00000100:00100000:1.0:1769074894.760522:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2512 00000100:00100000:1.0:1769074894.760528:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625262336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760560:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625262336:12345-0@lo:400:kworker.0 Request processed in 32us (559us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074894.760566:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2512 00000100:00100000:1.0:1769074894.760578:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2513 00000100:00100000:1.0:1769074894.760584:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625262848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760626:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625262848:12345-0@lo:400:kworker.0 Request processed in 42us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074894.760633:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2513 00000100:00080000:1.0:1769074894.760657:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074894.760673:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625262336:0@lo:400:kworker.0 00000100:00100000:1.0:1769074894.760680:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625262848:0@lo:400:kworker.0 00000100:00100000:2.0:1769074894.760977:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074894.761013:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625261952:0@lo:400:kworker.0 00000100:00100000:2.0:1769074894.761041:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074894.761061:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625262464:0@lo:400:kworker.0 00000100:00100000:2.0:1769074894.761093:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074894.761311:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074894.761338:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625262592:0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.761365:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074894.761402:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625261568:0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.761414:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1769074894.761421:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625261696:0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.761444:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074894.761464:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625262208:0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.761484:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074894.761504:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625262720:0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.761525:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074894.765456:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262592 00000100:00100000:3.0:1769074894.765477:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262592 00000100:00100000:3.0:1769074894.765564:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625261696 00000100:00100000:3.0:1769074894.765568:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625261696 00000100:00100000:3.0:1769074894.765596:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262720 00000100:00100000:3.0:1769074894.765600:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262720 00000100:00100000:3.0:1769074894.765648:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1586 00000100:00100000:3.0:1769074894.765660:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625262592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.765852:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3649:x1855007625262592:12345-0@lo:400:kworker.0 Request processed in 150us (4440us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.765869:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1586 00000100:00100000:3.0:1769074894.765890:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1587 00000100:00100000:3.0:1769074894.765896:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625261696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.765930:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625261696:12345-0@lo:400:kworker.0 Request processed in 35us (4485us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.765936:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1587 00000100:00100000:3.0:1769074894.765947:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1588 00000100:00100000:3.0:1769074894.765952:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625262720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.765981:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3649:x1855007625262720:12345-0@lo:400:kworker.0 Request processed in 30us (4455us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.765987:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1588 00000100:00100000:3.0:1769074894.766009:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625261952 00000100:00100000:3.0:1769074894.766014:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625261952 00000100:00100000:3.0:1769074894.766049:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262080 00000100:00100000:3.0:1769074894.766053:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262080 00000100:00100000:3.0:1769074894.766072:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262208 00000100:00100000:3.0:1769074894.766075:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262208 00000100:00100000:3.0:1769074894.766107:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2514 00000100:00100000:3.0:1769074894.766114:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625261952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766156:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625261952:12345-0@lo:400:kworker.0 Request processed in 43us (5113us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.766163:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2514 00000100:00100000:3.0:1769074894.766180:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2515 00000100:00100000:3.0:1769074894.766188:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625262080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766224:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625262080:12345-0@lo:400:kworker.0 Request processed in 38us (4913us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.766230:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2515 00000100:00100000:3.0:1769074894.766242:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2516 00000100:00100000:3.0:1769074894.766247:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625262208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766290:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625262208:12345-0@lo:400:kworker.0 Request processed in 43us (4804us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.766296:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2516 00000100:00100000:3.0:1769074894.766370:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625261696:0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766387:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625262208:0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766401:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625262720:0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766413:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625262080:0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.766425:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625262592:0@lo:400:kworker.0 00000100:00080000:3.0:1769074894.766472:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074894.766478:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074894.766482:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074894.766485:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074894.776879:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625261952:0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.779606:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625261440 00000100:00100000:3.0:1769074894.779628:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625261440 00000100:00100000:3.0:1769074894.779716:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262464 00000100:00100000:3.0:1769074894.779730:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262464 00000100:00100000:3.0:1769074894.779775:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2757 00000100:00100000:0.0:1769074894.779788:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2758 00000100:00100000:3.0:1769074894.779792:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625261440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074894.779830:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625262464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074894.780112:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625261440:12345-0@lo:400:kworker.0 Request processed in 322us (19133us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074894.780122:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2757 00000100:00100000:0.0:1769074894.780296:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625262464:12345-0@lo:400:kworker.0 Request processed in 496us (19200us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074894.780304:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2758 00000100:00100000:2.0:1769074894.781165:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625261440:0@lo:400:kworker.0 00000100:00100000:2.0:1769074894.781184:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625262464:0@lo:400:kworker.0 00010000:00010000:2.0:1769074895.263439:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074895.815337:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074895.815397:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074895.815417:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051520 00000100:00100000:2.0:1769074895.815438:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051520 00000100:00100000:2.0:1769074895.815523:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2517 00000100:00100000:2.0:1769074895.815532:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007617051520:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074895.815610:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007617051520:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 78us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074895.815619:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2517 00000100:00080000:3.0:1769074895.815768:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074895.815862:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051776 00000100:00100000:0.0:1769074895.815869:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051776 00000100:00100000:0.0:1769074895.815913:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1589 00000100:00100000:0.0:1769074895.815922:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617051776:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074895.815960:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617051776:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 39us (562us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074895.815961:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074895.815967:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1589 00000100:00100000:3.0:1769074895.816000:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051392 00000100:00100000:3.0:1769074895.816005:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051392 00000100:00100000:1.0:1769074895.816010:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074895.816041:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 391 00000100:00100000:3.0:1769074895.816049:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617051392:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074895.816101:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00080000:2.0:1769074895.816111:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074895.816134:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051904 00000100:00100000:2.0:1769074895.816139:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051904 00000100:00100000:2.0:1769074895.816182:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1590 00000100:00100000:2.0:1769074895.816188:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617051904:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074895.816220:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547598700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617051904:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (120us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074895.816226:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1590 00000100:00080000:1.0:1769074895.816370:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074895.816844:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617051392:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 793us (880us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074895.816853:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 391 00000100:00100000:0.0:1769074895.819370:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617051648 00000100:00100000:0.0:1769074895.819387:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617051648 00000100:00080000:1.0:1769074895.819967:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074895.820082:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2518 00000100:00100000:0.0:1769074895.820098:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617051648:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074895.820183:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617051648:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 89us (4175us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074895.829226:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2518 00010000:00010000:1.0:1769074896.285132:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074896.799407:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625262976:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074896.799485:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074896.799551:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625262976 00000100:00100000:1.0:1769074896.799559:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625262976 00000100:00100000:1.0:1769074896.799604:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2759 00000100:00100000:1.0:1769074896.799612:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625262976:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074896.799645:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:3.0:1769074896.799663:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625263104:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074896.799690:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625262976:12345-0@lo:41:osp-pre-1-0.0 Request processed in 78us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074896.799692:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074896.799697:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2759 00000100:00100000:1.0:1769074896.799715:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263104 00000100:00100000:1.0:1769074896.799718:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263104 00000100:00100000:1.0:1769074896.799741:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2760 00000100:00100000:1.0:1769074896.799747:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625263104:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074896.799768:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:1.0:1769074896.799830:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625263104:12345-0@lo:41:osp-pre-0-1.0 Request processed in 58us (112us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074896.799837:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2760 00000100:00100000:1.0:1769074896.799881:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625262976:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074896.799976:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625263104:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1769074897.309686:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074897.310071:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625263232:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074897.310144:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074897.311170:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625263360:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074897.311212:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074897.311856:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625263488:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074897.312784:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263232 00000100:00100000:3.0:1769074897.312793:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263232 00000100:00100000:3.0:1769074897.312842:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263360 00000100:00100000:3.0:1769074897.312845:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263360 00000100:00100000:3.0:1769074897.312860:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1264 00000100:00100000:3.0:1769074897.312867:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625263232:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769074897.312892:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074897.312935:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625263232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (2791us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074897.312942:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1264 00000100:00100000:3.0:1769074897.312958:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1265 00000100:00100000:3.0:1769074897.312964:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625263360:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1769074897.312988:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074897.313019:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625263360:12345-0@lo:13:osp-pre-0-1.0 Request processed in 56us (1807us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074897.313023:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1265 00000100:00080000:3.0:1769074897.313037:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074897.313040:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074897.313099:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625263232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074897.313708:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074897.313749:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263488 00000100:00100000:3.0:1769074897.313754:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263488 00000100:00100000:3.0:1769074897.313785:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1266 00000100:00100000:3.0:1769074897.313792:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625263488:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074897.313804:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074897.313860:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1855007625263488:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (153us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074897.313866:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1266 00000100:00080000:3.0:1769074897.313881:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074897.313906:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625263488:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074897.314273:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625263360:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074897.823484:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074897.823507:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625263616:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074897.823575:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074897.825819:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263616 00000100:00100000:0.0:1769074897.825902:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263616 00000100:00100000:2.0:1769074897.827093:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1267 00000100:00100000:2.0:1769074897.827102:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625263616:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074897.827126:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074897.827228:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625263616:12345-0@lo:13:osp-pre-1-1.0 Request processed in 126us (3652us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074897.827236:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1267 00000100:00100000:1.0:1769074897.827426:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625263616:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1769074897.827445:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074898.333077:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074899.357880:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1769074899.361112:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874644 is committed 00000001:00080000:3.0:1769074899.369659:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841312 is committed 00000100:00100000:0.0:1769074899.874755:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625263744:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875108:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625264128:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875206:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875267:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656db800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625264640:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875290:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875308:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625265152:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875325:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875349:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074899.875353:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625264256:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875371:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875385:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625264768:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875401:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875417:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625265280:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875435:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875462:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625263872:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875486:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875504:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625264384:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875519:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875534:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625264896:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875551:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875567:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1769074899.875571:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625264000:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875588:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.875601:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625264512:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.875617:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074899.876014:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074899.876073:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264128 00000100:00100000:0.0:1769074899.876084:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264128 00000100:00100000:0.0:1769074899.876149:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265152 00000100:00100000:0.0:1769074899.876151:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265152 00000100:00100000:0.0:1769074899.876172:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264256 00000100:00100000:0.0:1769074899.876187:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264256 00000100:00100000:0.0:1769074899.876218:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1591 00000100:00100000:0.0:1769074899.876228:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625264128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876298:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625264128:12345-0@lo:400:kworker.0 Request processed in 72us (1092us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876306:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1591 00000100:00100000:0.0:1769074899.876327:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1592 00000100:00100000:0.0:1769074899.876334:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625265152:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876364:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625265152:12345-0@lo:400:kworker.0 Request processed in 32us (1038us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876371:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1592 00000100:00100000:0.0:1769074899.876385:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1593 00000100:00100000:0.0:1769074899.876392:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625264256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876420:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625264256:12345-0@lo:400:kworker.0 Request processed in 29us (1048us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876426:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1593 00000100:00100000:0.0:1769074899.876447:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264640 00000100:00100000:0.0:1769074899.876451:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264640 00000100:00100000:0.0:1769074899.876476:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264768 00000100:00100000:0.0:1769074899.876479:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264768 00000100:00100000:0.0:1769074899.876494:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265280 00000100:00100000:0.0:1769074899.876497:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265280 00000100:00100000:0.0:1769074899.876508:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264384 00000100:00100000:0.0:1769074899.876511:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264384 00000100:00100000:0.0:1769074899.876522:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264000 00000100:00100000:0.0:1769074899.876524:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264000 00000100:00100000:0.0:1769074899.876533:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264512 00000100:00100000:0.0:1769074899.876535:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264512 00000100:00100000:0.0:1769074899.876547:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2519 00000100:00100000:0.0:1769074899.876553:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625264640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876582:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625264640:12345-0@lo:400:kworker.0 Request processed in 30us (1292us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876589:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2519 00000100:00100000:0.0:1769074899.876601:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2520 00000100:00100000:0.0:1769074899.876606:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625264768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876630:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625264768:12345-0@lo:400:kworker.0 Request processed in 24us (1227us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876647:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2520 00000100:00100000:0.0:1769074899.876658:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2521 00000100:00100000:0.0:1769074899.876663:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625265280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876691:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625265280:12345-0@lo:400:kworker.0 Request processed in 29us (1255us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876697:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2521 00000100:00100000:0.0:1769074899.876706:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2522 00000100:00100000:0.0:1769074899.876710:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625264384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.876730:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625264384:12345-0@lo:400:kworker.0 Request processed in 20us (1210us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.876736:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2522 00000100:00100000:0.0:1769074899.876744:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2523 00000100:00100000:0.0:1769074899.876750:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625264000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877015:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625265024:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877055:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074899.877103:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625264384:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877143:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265024 00000100:00100000:2.0:1769074899.877146:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265024 00000100:00100000:2.0:1769074899.877192:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1594 00000100:00100000:2.0:1769074899.877200:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625265024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877241:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625265024:12345-0@lo:400:kworker.0 Request processed in 42us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074899.877249:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1594 00000100:00100000:2.0:1769074899.877277:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2524 00000100:00100000:2.0:1769074899.877286:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625264512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877320:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625264512:12345-0@lo:400:kworker.0 Request processed in 36us (1701us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074899.877325:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2524 00000100:00100000:2.0:1769074899.877360:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263872 00000100:00100000:2.0:1769074899.877364:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263872 00000100:00100000:2.0:1769074899.877391:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625264896 00000100:00100000:2.0:1769074899.877395:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625264896 00000100:00100000:2.0:1769074899.877426:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2761 00000100:00100000:2.0:1769074899.877432:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625263872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877468:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625263872:12345-0@lo:400:kworker.0 Request processed in 35us (1981us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074899.877473:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2761 00000100:00100000:2.0:1769074899.877485:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2762 00000100:00100000:2.0:1769074899.877490:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625264896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877533:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625264896:12345-0@lo:400:kworker.0 Request processed in 42us (1979us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074899.877539:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2762 00000100:00100000:2.0:1769074899.877634:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625264512:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877644:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 21474841312 gen 2 00000004:00080000:2.0:1769074899.877649:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b547599880, transno 21474841312 00000100:00100000:2.0:1769074899.877661:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625265024:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877673:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549202680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625263872:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.877684:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625264896:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878034:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625264000:12345-0@lo:400:kworker.0 Request processed in 1282us (2443us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.878043:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2523 00000100:00100000:0.0:1769074899.878075:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625263744 00000100:00100000:0.0:1769074899.878082:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625263744 00000100:00100000:0.0:1769074899.878119:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:0.0:1769074899.878127:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625263744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878186:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625263744:12345-0@lo:400:kworker.0 Request processed in 49us (2160us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074899.878193:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00080000:0.0:1769074899.878230:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074899.878236:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074899.878241:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074899.878244:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074899.878294:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625264256:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878310:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625264768:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878323:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625265280:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878337:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625264128:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878350:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656db800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625264640:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878363:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 17179874644 gen 2 00000004:00080000:0.0:1769074899.878368:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5476f9880, transno 17179874643 00000004:00080000:0.0:1769074899.878379:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b5476fbb80, transno 17179874644 00000100:00100000:0.0:1769074899.878384:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625265152:0@lo:400:kworker.0 00000100:00100000:0.0:1769074899.878530:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549201500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625263744:0@lo:400:kworker.0 00000100:00100000:2.0:1769074899.879103:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549203b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625264000:0@lo:400:kworker.0 00010000:00010000:0.0:1769074900.381107:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074900.938779:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074900.938866:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074900.938909:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074900.938941:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074900.939007:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074900.939031:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052288 00000100:00100000:2.0:1769074900.939040:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052288 00000100:00100000:2.0:1769074900.939091:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052160 00000100:00100000:2.0:1769074900.939094:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052160 00000100:00100000:2.0:1769074900.939115:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2525 00000100:00100000:2.0:1769074900.939123:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617052288:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074900.939206:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617052288:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 83us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074900.939213:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2525 00000100:00100000:2.0:1769074900.939231:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2526 00000100:00100000:2.0:1769074900.939238:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007617052160:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074900.939262:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2407:x1855007617052160:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074900.939268:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2526 00000100:00100000:3.0:1769074900.939867:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052544 00000100:00100000:3.0:1769074900.939873:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052544 00000100:00100000:3.0:1769074900.939913:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052416 00000100:00100000:1.0:1769074900.940003:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1595 00000100:00100000:1.0:1769074900.940014:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617052544:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074900.940058:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617052544:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 47us (1149us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074900.940065:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1595 00000100:00100000:3.0:1769074900.940201:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052416 00000100:00100000:3.0:1769074900.940236:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1596 00000100:00100000:3.0:1769074900.940243:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617052416:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074900.940274:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617052416:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 32us (1267us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074900.940280:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1596 00000100:00100000:0.0:1769074900.940621:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052032 00000100:00100000:0.0:1769074900.940627:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052032 00000100:00100000:0.0:1769074900.940656:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 393 00000100:00100000:0.0:1769074900.940662:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b09850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617052032:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074900.940710:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b09850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617052032:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 47us (1942us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074900.940717:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 393 00000100:00080000:0.0:1769074900.940752:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074900.940756:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074900.940758:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074900.940761:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1769074901.413649:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074901.918663:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074901.918688:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625265408:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074901.918755:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074901.919001:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074901.919013:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625265536:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074901.919043:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074901.919119:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265408 00000100:00100000:0.0:1769074901.919127:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265408 00000100:00100000:0.0:1769074901.919183:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265536 00000100:00100000:0.0:1769074901.919186:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265536 00000100:00100000:0.0:1769074901.919208:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2763 00000100:00100000:0.0:1769074901.919215:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625265408:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074901.919246:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:0.0:1769074901.919290:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625265408:12345-0@lo:41:osp-pre-1-0.0 Request processed in 74us (534us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074901.919297:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2763 00000100:00100000:0.0:1769074901.919313:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2764 00000100:00100000:0.0:1769074901.919319:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625265536:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074901.919334:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:0.0:1769074901.919365:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625265536:12345-0@lo:41:osp-pre-0-1.0 Request processed in 46us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074901.919370:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2764 00000100:00100000:0.0:1769074901.919424:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625265536:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074901.919789:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625265408:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074902.429249:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625265664:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074902.429319:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074902.429393:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265664 00000100:00100000:1.0:1769074902.429400:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265664 00000100:00100000:1.0:1769074902.429449:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1268 00000100:00100000:1.0:1769074902.429458:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625265664:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074902.429502:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074902.429546:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625265664:12345-0@lo:13:osp-pre-0-1.0 Request processed in 89us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074902.429553:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1268 00000100:00080000:1.0:1769074902.429575:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074902.429612:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625265664:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074902.430350:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074902.430362:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625265920:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074902.430398:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074902.430428:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265920 00000100:00100000:1.0:1769074902.430433:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265920 00000100:00100000:1.0:1769074902.430466:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1269 00000100:00100000:1.0:1769074902.430473:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625265920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074902.430504:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074902.430539:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625265920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074902.430545:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1269 00000100:00100000:1.0:1769074902.430566:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625265792:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074902.430592:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1769074902.430617:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074902.430637:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625265920:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074902.430656:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625265792 00000100:00100000:1.0:1769074902.430659:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625265792 00000100:00100000:1.0:1769074902.430686:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1270 00000100:00100000:1.0:1769074902.430692:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625265792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074902.430708:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074902.430741:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625265792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (149us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074902.430747:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1270 00000100:00080000:1.0:1769074902.430764:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074902.430784:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625265792:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1769074902.431450:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000004:2.0:1769074902.834670:0:38802:0:(tgt_mount.c:2194:server_getattr()) lustre-OST0001: root_inode from vfsmnt ino=2, dev=0 00000020:00000004:2.0:1769074902.834681:0:38802:0:(tgt_mount.c:2225:is_cmd_supported()) ioctl cmd=c0185879 00000020:00000004:3.0:1769074902.838199:0:38804:0:(tgt_mount.c:2194:server_getattr()) lustre-OST0000: root_inode from vfsmnt ino=2, dev=0 00000020:00000004:3.0:1769074902.838211:0:38804:0:(tgt_mount.c:2225:is_cmd_supported()) ioctl cmd=c0185879 00000100:00100000:3.0:1769074902.950518:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5444a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625266048:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1769074902.950590:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074902.950662:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266048 00000100:00100000:3.0:1769074902.950670:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266048 00000100:00100000:3.0:1769074902.950723:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1271 00000100:00100000:3.0:1769074902.950732:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625266048:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074902.950755:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074902.950844:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625266048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 113us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074902.950855:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1271 00000100:00080000:3.0:1769074902.950878:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074902.950913:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5444a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625266048:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1769074903.462433:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074904.486463:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074904.989226:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625266304:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989297:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989359:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625266816:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989385:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989406:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625267328:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989426:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989451:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:3.0:1769074904.989457:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625266176:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989485:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989508:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625266688:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989544:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989577:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625267200:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989598:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989613:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625267712:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989630:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989656:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625266432:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989677:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989693:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625266944:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989710:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989724:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625267456:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989743:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989760:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1769074904.989765:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625266560:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989786:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989801:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625267072:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989853:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989865:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625267584:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.989883:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074904.989916:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266816 00000100:00100000:3.0:1769074904.989924:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266816 00000100:00100000:3.0:1769074904.989968:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267200 00000100:00100000:3.0:1769074904.989971:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267200 00000100:00100000:3.0:1769074904.989984:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267712 00000100:00100000:3.0:1769074904.989987:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267712 00000100:00100000:3.0:1769074904.989998:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266432 00000100:00100000:3.0:1769074904.990001:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266432 00000100:00100000:3.0:1769074904.990010:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266944 00000100:00100000:3.0:1769074904.990012:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266944 00000100:00100000:3.0:1769074904.990023:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267072 00000100:00100000:3.0:1769074904.990025:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267072 00000100:00100000:3.0:1769074904.990039:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2527 00000100:00100000:3.0:1769074904.990047:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625266816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990112:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625266816:12345-0@lo:400:kworker.0 Request processed in 66us (725us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990120:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2527 00000100:00100000:3.0:1769074904.990137:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2528 00000100:00100000:3.0:1769074904.990143:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625267200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990171:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625267200:12345-0@lo:400:kworker.0 Request processed in 28us (572us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990178:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2528 00000100:00100000:3.0:1769074904.990189:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2529 00000100:00100000:3.0:1769074904.990194:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625267712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990219:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625267712:12345-0@lo:400:kworker.0 Request processed in 25us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990224:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2529 00000100:00100000:3.0:1769074904.990236:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2530 00000100:00100000:3.0:1769074904.990242:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625266432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990270:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625266432:12345-0@lo:400:kworker.0 Request processed in 29us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990276:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2530 00000100:00100000:3.0:1769074904.990286:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2531 00000100:00100000:3.0:1769074904.990292:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625266944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990318:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a272a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625266944:12345-0@lo:400:kworker.0 Request processed in 26us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990323:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2531 00000100:00100000:3.0:1769074904.990333:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2532 00000100:00100000:3.0:1769074904.990339:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625267072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990363:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625267072:12345-0@lo:400:kworker.0 Request processed in 24us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990368:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2532 00000100:00100000:3.0:1769074904.990386:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266304 00000100:00100000:3.0:1769074904.990390:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266304 00000100:00100000:3.0:1769074904.990411:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267328 00000100:00100000:3.0:1769074904.990413:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267328 00000100:00100000:3.0:1769074904.990430:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2765 00000100:00100000:3.0:1769074904.990436:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625266304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990465:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625266304:12345-0@lo:400:kworker.0 Request processed in 29us (1168us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990470:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2765 00000100:00100000:3.0:1769074904.990481:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2766 00000100:00100000:3.0:1769074904.990486:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625267328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990511:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a270380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625267328:12345-0@lo:400:kworker.0 Request processed in 25us (1084us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990517:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2766 00000100:00100000:3.0:1769074904.990535:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266176 00000100:00100000:3.0:1769074904.990539:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266176 00000100:00100000:3.0:1769074904.990562:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:3.0:1769074904.990568:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b08450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625266176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990596:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b08450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625266176:12345-0@lo:400:kworker.0 Request processed in 28us (1110us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990601:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:3.0:1769074904.990630:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266688 00000100:00100000:3.0:1769074904.990633:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266688 00000100:00100000:3.0:1769074904.990654:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267456 00000100:00100000:3.0:1769074904.990656:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267456 00000100:00100000:3.0:1769074904.990669:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625266560 00000100:00100000:3.0:1769074904.990671:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625266560 00000100:00100000:3.0:1769074904.990683:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267584 00000100:00100000:3.0:1769074904.990686:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267584 00000100:00100000:3.0:1769074904.990701:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1597 00000100:00100000:3.0:1769074904.990707:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625266688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990735:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625266688:12345-0@lo:400:kworker.0 Request processed in 29us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990741:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1597 00000100:00100000:3.0:1769074904.990751:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1598 00000100:00100000:3.0:1769074904.990756:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625267456:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990781:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625267456:12345-0@lo:400:kworker.0 Request processed in 24us (1036us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990786:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1598 00000100:00100000:3.0:1769074904.990795:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1599 00000100:00100000:3.0:1769074904.990801:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625266560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990852:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625266560:12345-0@lo:400:kworker.0 Request processed in 51us (1064us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990858:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1599 00000100:00100000:3.0:1769074904.990869:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1600 00000100:00100000:3.0:1769074904.990874:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625267584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.990900:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625267584:12345-0@lo:400:kworker.0 Request processed in 26us (1015us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074904.990905:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1600 00000100:00080000:3.0:1769074904.990947:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074904.990951:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074904.990954:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074904.990955:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769074904.990995:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625266560:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991007:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625267072:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991016:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625267584:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991025:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625266432:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991034:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625266944:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991043:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625267456:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991087:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625266176:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991111:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625266688:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991120:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625267200:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991131:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625267712:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991141:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625266304:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991150:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625266816:0@lo:400:kworker.0 00000100:00100000:3.0:1769074904.991159:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625267328:0@lo:400:kworker.0 00010000:00010000:0.0:1769074905.517606:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074906.055954:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074906.056007:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074906.056037:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074906.056080:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074906.056111:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074906.056140:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052928 00000100:00100000:2.0:1769074906.056149:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052928 00000100:00100000:2.0:1769074906.056231:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052800 00000100:00100000:2.0:1769074906.056235:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052800 00000100:00100000:2.0:1769074906.056261:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2533 00000100:00100000:2.0:1769074906.056271:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617052928:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074906.056347:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617052928:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 78us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074906.056354:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2533 00000100:00100000:2.0:1769074906.056371:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2534 00000100:00100000:2.0:1769074906.056376:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007617052800:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074906.056401:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:2408:x1855007617052800:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074906.056407:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2534 00000100:00080000:3.0:1769074906.056565:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074906.056569:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074906.057135:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617052672 00000100:00100000:0.0:1769074906.057143:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617052672 00000100:00100000:0.0:1769074906.057188:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 395 00000100:00100000:0.0:1769074906.057196:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b08050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617052672:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074906.057242:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b08050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617052672:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 47us (1298us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074906.057250:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 395 00000100:00100000:2.0:1769074906.057389:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053184 00000100:00100000:2.0:1769074906.057394:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053184 00000100:00080000:0.0:1769074906.057466:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074906.062133:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053056 00000100:00100000:1.0:1769074906.062151:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053056 00000100:00080000:0.0:1769074906.062274:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074906.066669:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1601 00000100:00100000:2.0:1769074906.066692:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617053184:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074906.066776:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617053184:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 95us (10737us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074906.066783:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1601 00000100:00100000:1.0:1769074906.067713:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1602 00000100:00100000:1.0:1769074906.067726:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617053056:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074906.067782:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617053056:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 60us (11669us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074906.067789:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1602 00010000:00010000:0.0:1769074906.525733:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074907.037100:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074907.037126:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625267840:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074907.037197:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074907.037296:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267840 00000100:00100000:0.0:1769074907.037305:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267840 00000100:00100000:0.0:1769074907.037367:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2767 00000100:00100000:0.0:1769074907.037376:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625267840:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074907.037419:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445367808 avail 1299128320 00000100:00100000:0.0:1769074907.037468:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625267840:12345-0@lo:41:osp-pre-0-1.0 Request processed in 93us (270us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074907.037476:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2767 00000100:00100000:2.0:1769074907.037642:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625267840:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074907.039964:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625267968:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074907.040036:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074907.040165:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625267968 00000100:00100000:2.0:1769074907.040173:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625267968 00000100:00100000:2.0:1769074907.040228:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2768 00000100:00100000:2.0:1769074907.040237:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625267968:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074907.040270:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446125568 avail 1299886080 00000100:00100000:2.0:1769074907.040330:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625267968:12345-0@lo:41:osp-pre-1-0.0 Request processed in 94us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074907.040338:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2768 00000100:00100000:3.0:1769074907.040431:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476faa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625267968:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074907.452714:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.453256:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053312 00000100:00100000:3.0:1769074907.453268:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053312 00000100:00100000:3.0:1769074907.453339:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13635 00000100:00100000:3.0:1769074907.453350:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:68429:x1855007617053312:12345-192.168.204.60@tcp:41:stat.0 00000004:00000024:3.0:1769074907.453400:0:10283:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7830949888 avail 7367905280 00000100:00100000:3.0:1769074907.453486:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:68429:x1855007617053312:12345-192.168.204.60@tcp:41:stat.0 Request processed in 138us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074907.453497:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13635 00000100:00100000:0.0:1769074907.499403:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.499756:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053440 00000100:00100000:3.0:1769074907.499767:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053440 00000100:00100000:3.0:1769074907.499854:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13636 00000100:00100000:3.0:1769074907.499864:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:68433:x1855007617053440:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074907.499884:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074907.499905:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55cc8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c96d expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.499967:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.499980:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.499992:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500005:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500012:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500018:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500070:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500077:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500083:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500090:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500095:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.500098:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500105:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500115:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074907.500118:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.500123:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500129:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55ccf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500142:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500147:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500152:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500157:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febaa00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.500159:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500169:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500176:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8f26d1c0 (ffff89b64a19d600) 00010000:00010000:3.0:1769074907.500179:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.500181:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500187:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55cd6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.500217:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55cc8 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c96d expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.500226:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074907.500229:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.500233:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55cc8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c96d expref: 21 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.500241:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbc00, rc 0) 00000100:00100000:3.0:1769074907.500312:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:68433:x1855007617053440:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 449us (920us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074907.500318:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13636 00000100:00100000:2.0:1769074907.513796:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074907.516860:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053696 00000100:00100000:0.0:1769074907.516887:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053696 00000100:00100000:0.0:1769074907.516955:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13637 00000100:00100000:0.0:1769074907.516964:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+20:68433:x1855007617053696:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:0.0:1769074907.517021:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074907.517056:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd55933 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c87f expref: 20 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.517081:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:0.0:1769074907.517086:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074907.517146:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.517159:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.517168:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.517183:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.517189:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.517194:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000040:00080000:0.0:1769074907.518129:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].87, 2168 off94976 00000040:00080000:0.0:1769074907.518207:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000405:0x2:0x0].9, 2368 off52096 00000004:00080000:0.0:1769074907.518215:0:6514:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b67fc18480 with version 23 00000004:00080000:1.0:1769074907.518487:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 23 rpc_version 23 00000020:00100000:1.0:1769074907.518523:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1769074907.518533:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b5476fa680 x1855007625268224/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1769074907.518555:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625268224:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074907.518618:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074907.518668:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5437ba000 going to sleep for 11 seconds 00000100:00100000:2.0:1769074907.521887:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268224 00000100:00100000:2.0:1769074907.521903:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268224 00000100:00100000:2.0:1769074907.521967:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2769 00000100:00100000:2.0:1769074907.521977:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625268224:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1769074907.522524:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625268224:12345-0@lo:1000:osp_up1-0.0 Request processed in 548us (3904us total) trans 4294968372 rc 0/0 00000100:00100000:2.0:1769074907.522533:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2769 00000100:00100000:1.0:1769074907.523996:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476fa680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625268224:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1769074907.524292:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,1 mode: --/PW res: [0x200000405:0x20c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524313:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524325:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524344:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524352:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524357:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524391:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524398:0:6514:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.524406:0:6514:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6514 initiator: MDT0 00000100:00100000:3.0:1769074907.524427:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6514:1855007625268352:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1769074907.524478:0:6514:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074907.524521:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b649221480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074907.525395:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268352 00000100:00100000:2.0:1769074907.525405:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268352 00000100:00100000:2.0:1769074907.525462:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2770 00000100:00100000:2.0:1769074907.525472:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6514:x1855007625268352:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1769074907.525491:0:7791:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074907.525511:0:7791:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 2/0,0 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7fe2323adfd55cf9 expref: -99 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.525535:0:7791:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7fe2323adfd55cf9 expref: 7 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.525548:0:7791:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd55cf9 expref: 7 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.525556:0:7791:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074907.525559:0:7791:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074907.525562:0:7791:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074907.525622:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6514:x1855007625268352:12345-0@lo:101:mdt00_002.0 Request processed in 152us (1143us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074907.525629:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2770 00000100:00100000:3.0:1769074907.526501:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6514:1855007625268352:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1769074907.526521:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.526532:0:6514:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1769074907.526558:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000078003a79 reply state 000000005f45baa4 transno 8589944139 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526573:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526579:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xa:0x0].0x0 (ffff89b64a19dc00) 00010000:00010000:3.0:1769074907.526582:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074907.526585:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1769074907.526590:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.526597:0:6514:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000004:00080000:3.0:1769074907.526606:0:6514:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000405:0xa:0x0] 00000004:00010000:3.0:1769074907.526613:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000078003a79 reply state 000000005f45baa4 transno 8589944139 ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526621:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526627:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20c:0x0].0x0 (ffff89b64a19d300) 00010000:00010000:3.0:1769074907.526630:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074907.526632:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1769074907.526637:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20c:0x0]: returning size 4096 00010000:00010000:3.0:1769074907.526648:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526653:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526658:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526663:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526669:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.526673:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526680:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526689:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074907.526691:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.526695:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526701:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134200/0x7fe2323adfd55cdd lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00000004:00010000:3.0:1769074907.526719:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000078003a79 reply state 000000005f45baa4 transno 8589944139 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526727:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.526732:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8f26d1c0 (ffff89b67a317000) 00010000:00010000:3.0:1769074907.526734:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1769074907.526736:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 2 00000100:00100000:3.0:1769074907.526935:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:68433:x1855007617053696:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 9896us (13076us total) trans 8589944139 rc 0/0 00000100:00100000:3.0:1769074907.526945:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13637 00010000:00010000:0.0:1769074907.549188:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074907.549411:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074907.549447:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625268480:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074907.549707:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074907.549716:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625268608:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074907.549782:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074907.549873:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268608 00000100:00100000:3.0:1769074907.549882:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268608 00000100:00100000:3.0:1769074907.549932:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1272 00000100:00100000:3.0:1769074907.549941:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625268608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769074907.549968:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074907.550012:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625268608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 72us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074907.550018:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1272 00000100:00100000:3.0:1769074907.550081:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625268608:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1769074907.550168:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074907.550265:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074907.550406:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769074907.550416:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625268736:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074907.550455:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074907.550742:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268480 00000100:00100000:1.0:1769074907.550747:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268480 00000100:00100000:1.0:1769074907.550790:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268736 00000100:00100000:1.0:1769074907.550794:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268736 00000100:00100000:1.0:1769074907.550830:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1273 00000100:00100000:1.0:1769074907.550837:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625268480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074907.550855:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074907.550892:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625268480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 56us (627us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074907.550899:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1273 00000100:00100000:1.0:1769074907.550914:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1274 00000100:00100000:1.0:1769074907.550920:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625268736:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074907.550930:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074907.550955:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625268736:12345-0@lo:13:osp-pre-0-1.0 Request processed in 35us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074907.550963:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1274 00000100:00100000:1.0:1769074907.550995:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5476f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625268480:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1769074907.551064:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074907.551127:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625268736:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074907.591686:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.591988:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053824 00000100:00100000:3.0:1769074907.592003:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053824 00000100:00100000:3.0:1769074907.592104:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13638 00000100:00100000:3.0:1769074907.592117:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+21:68436:x1855007617053824:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074907.592148:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074907.592183:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d07 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c974 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.592285:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55d0e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.592303:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55d0e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.592317:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55d0e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00000004:00200000:3.0:1769074907.592382:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1769074907.592387:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074907.592413:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55d0e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT0 00010000:00010000:3.0:1769074907.592458:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d07 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c974 expref: 23 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.592482:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55d0e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c974 expref: 22 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.592493:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074907.592496:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769074907.592500:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febaa00, rc 0) 00000100:00100000:3.0:1769074907.592580:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:68436:x1855007617053824:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 465us (908us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074907.592588:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13638 00000100:00100000:0.0:1769074907.599100:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.599274:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617053952 00000100:00100000:3.0:1769074907.599286:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617053952 00000100:00100000:3.0:1769074907.599363:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13639 00000100:00100000:3.0:1769074907.599375:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+22:68436:x1855007617053952:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074907.599415:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074907.599444:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d15 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c97b expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.599704:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.599729:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.599741:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.599756:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.599764:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.599770:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600129:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd55d2a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600139:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd55d2a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600146:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd55d2a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074907.600223:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20c:0x0]: returning size 4096 00000004:00200000:3.0:1769074907.600229:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1769074907.600250:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd55d2a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600262:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600269:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600275:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600282:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600293:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.600296:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600303:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600315:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074907.600318:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.600323:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600329:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba400/0x7fe2323adfd55d1c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600342:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600347:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600352:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600357:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.600359:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600364:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600371:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8f26d1c0 (ffff89b67a317000) 00010000:00010000:3.0:1769074907.600373:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.600376:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600381:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d23 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.600410:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d15 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c97b expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.600425:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febba00/0x7fe2323adfd55d2a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c97b expref: 23 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.600432:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20c:0x0].0x0 (ffff89b64a19d300) 00010000:00010000:3.0:1769074907.600434:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1769074907.600436:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febba00, rc 0) 00000100:00100000:3.0:1769074907.600592:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:68436:x1855007617053952:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1218us (1508us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074907.600604:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13639 00000100:00100000:2.0:1769074907.612276:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.612369:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617054080 00000100:00100000:3.0:1769074907.612379:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617054080 00000100:00100000:3.0:1769074907.612445:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13640 00000100:00100000:3.0:1769074907.612455:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:68436:x1855007617054080:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074907.612475:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074907.612496:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d31 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c982 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.612565:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d38 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0xa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.612580:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d38 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.612591:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d38 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:3.0:1769074907.612641:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd0:0xa:0x0]: returning size 4096 00000004:00200000:3.0:1769074907.612646:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1769074907.612679:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d31 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c982 expref: 25 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.612696:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb000/0x7fe2323adfd55d38 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xa:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c982 expref: 24 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.612704:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xa:0x0].0x0 (ffff89b64a19dc00) 00010000:00010000:3.0:1769074907.612707:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1769074907.612710:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb000, rc 0) 00000100:00100000:3.0:1769074907.612770:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:68436:x1855007617054080:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 316us (505us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074907.612777:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13640 00000100:00100000:1.0:1769074907.617110:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.617239:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617054208 00000100:00100000:3.0:1769074907.617250:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617054208 00000100:00100000:3.0:1769074907.617357:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13641 00000100:00100000:3.0:1769074907.617369:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:68436:x1855007617054208:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074907.617390:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074907.617410:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba400/0x7fe2323adfd55d3f lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c989 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.617462:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.617473:0:10283:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 2/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd55cf9 expref: 6 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.617480:0:10283:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 4/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.617510:0:10283:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7fe2323adfd55cf9 expref: 6 pid: 7791 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074907.617524:0:10283:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10283:1855007625268864:0@lo:104: 00000100:00100000:3.0:1769074907.617569:0:10283:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074907.617668:0:10283:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b649221480 going to sleep for 11 seconds 00000100:00100000:2.0:1769074907.618521:0:6496:0:(service.c:153:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1769074907.618537:0:6496:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268864 00000100:00100000:2.0:1769074907.618544:0:6496:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268864 00000100:00100000:2.0:1769074907.618636:0:6496:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:2.0:1769074907.618645:0:6496:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54501d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:10283:x1855007625268864:12345-0@lo:104: 00010000:00010000:2.0:1769074907.618660:0:6496:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.618727:0:6496:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1769074907.618732:0:6496:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54501d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:10283:x1855007625268864:12345-0@lo:104: Request processed in 89us (1162us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074907.618739:0:6496:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:3.0:1769074907.618851:0:10283:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10283:1855007625268864:0@lo:104: 00010000:00010000:3.0:1769074907.618870:0:10283:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074907.618918:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1769074907.618925:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.618938:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba000 already unused, calling callback (ffffffffc1908290) 00010000:00010000:0.0:1769074907.618944:0:8413:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1769074907.618952:0:8413:0:(mdt_handler.c:3944:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.618980:0:8413:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 2/0,0 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.618986:0:8413:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1769074907.619015:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 1/0,0 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.619023:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b67feba000/0x7fe2323adfd55cf9 lrc: 0/0,0 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd55d00 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.619041:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074907.619042:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074907.619052:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074907.619060:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074907.619070:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625268992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1769074907.619102:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074907.619130:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625268992 00000100:00100000:0.0:1769074907.619134:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625268992 00010000:00010000:0.0:1769074907.619152:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7fe2323adfd55cf9 expref: 7 pid: 7791 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074907.619172:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7232 00000100:00100000:0.0:1769074907.619178:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3652:x1855007625268992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1769074907.619189:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1769074907.625631:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968372 is committed 00010000:00010000:2.0:1769074907.625785:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d00 lrc: 0/0,0 mode: --/PW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7fe2323adfd55cf9 expref: 6 pid: 7791 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.625847:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074907.625853:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.625862:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.625870:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 4/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.625896:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074907.625981:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625268992:12345-0@lo:103:ldlm_bl.0 Request processed in 6801us (6877us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074907.625991:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7232 00000100:00100000:1.0:1769074907.626165:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67ff0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625268992:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1769074907.626519:0:10283:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074907.626528:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:0.0:1769074907.626605:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000405:0xa:0x0]: returning size 4096 00000004:00200000:0.0:1769074907.626611:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1769074907.626656:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67feba400/0x7fe2323adfd55d3f lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c989 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.626674:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c989 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.626682:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:0.0:1769074907.626684:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1769074907.626688:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb200, rc 0) 00000100:00100000:0.0:1769074907.626757:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68436:x1855007617054208:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 9391us (9656us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074907.626765:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13641 00000100:00100000:0.0:1769074907.629850:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.630088:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617054336 00000100:00100000:3.0:1769074907.630098:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617054336 00000100:00100000:3.0:1769074907.630147:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13642 00000100:00100000:3.0:1769074907.630155:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68436:x1855007617054336:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074907.630184:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074907.630204:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d4d lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c990 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.630263:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.630276:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630286:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630601:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074907.630611:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630618:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630679:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630686:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630693:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630699:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630704:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbe00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.630708:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630715:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630725:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:3.0:1769074907.630728:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.630732:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630738:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55d54 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630750:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630756:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630762:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630766:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074907.630769:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630774:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630780:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64a19d600) 00010000:00010000:3.0:1769074907.630782:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.630785:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.630790:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55d5b lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074907.631003:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d4d lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c990 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.631014:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:3.0:1769074907.631016:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074907.631020:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55d4d lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c990 expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074907.631034:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbc00, rc 0) 00000100:00100000:3.0:1769074907.631139:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68436:x1855007617054336:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 983us (1339us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074907.631147:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13642 00000100:00100000:2.0:1769074907.642528:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074907.642787:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617054464 00000100:00100000:3.0:1769074907.642798:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617054464 00000100:00100000:3.0:1769074907.642899:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13643 00000100:00100000:3.0:1769074907.643099:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b424e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:68436:x1855007617054464:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074907.646824:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074907.646873:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d62 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c997 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.646956:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55d69 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074907.646971:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55d69 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074907.646983:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55d69 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074907.647094:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20c:0x0]: returning size 4096 00000004:00200000:2.0:1769074907.647100:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1769074907.647119:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55d69 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074907.647147:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd55d62 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c997 expref: 26 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.647170:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88200/0x7fe2323adfd55d69 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c997 expref: 25 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.647570:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20c:0x0].0x0 (ffff89b64a19d300) 00010000:00010000:2.0:1769074907.647574:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074907.647577:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88200, rc 0) 00000100:00100000:2.0:1769074907.647656:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b424e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:68436:x1855007617054464:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 4747us (5136us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074907.647665:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13643 00000100:00100000:1.0:1769074907.660620:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074907.660702:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617054720 00000100:00100000:2.0:1769074907.660713:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617054720 00000100:00100000:2.0:1769074907.660776:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13644 00000100:00100000:2.0:1769074907.660787:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68436:x1855007617054720:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074907.660836:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074907.660839:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1769074907.660865:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55d46 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c989 expref: 8 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.660883:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074907.660893:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074907.660920:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55d70 lrc: 2/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c99e expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.661022:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 3/0,1 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074907.661036:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 3/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661054:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 3/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661071:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,1 mode: --/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074907.661077:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661083:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661492:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074907.661502:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661509:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:2.0:1769074907.661519:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 0 00010000:00010000:2.0:1769074907.661544:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 4/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661551:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 4/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661556:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 3/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661562:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 4/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661567:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074907.661571:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 4/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661577:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 5/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661586:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa900) 00010000:00010000:2.0:1769074907.661588:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074907.661591:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 3/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661596:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88600/0x7fe2323adfd55d77 lrc: 0/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000004:00010000:2.0:1769074907.661607:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000caa428ac reply state 00000000bb71dd63 transno 4294968373 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661616:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074907.661621:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa000) 00010000:00010000:2.0:1769074907.661624:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074907.661626:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074907.661654:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55d70 lrc: 0/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 10 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.661670:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 9 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.661676:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074907.661678:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074907.661681:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88000, rc 0) 00000100:00100000:2.0:1769074907.661746:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68436:x1855007617054720:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 961us (1137us total) trans 4294968373 rc 0/0 00000100:00100000:2.0:1769074907.661758:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13644 00000100:00100000:0.0:1769074907.670277:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074907.670384:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617054976 00000100:00100000:1.0:1769074907.670398:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617054976 00000100:00100000:1.0:1769074907.670459:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13645 00000100:00100000:1.0:1769074907.670469:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68436:x1855007617054976:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074907.670493:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074907.670512:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55d93 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9a5 expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074907.670624:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074907.670639:0:6514:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074907.670647:0:6514:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 4/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.670678:0:6514:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074907.670690:0:6514:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 11 pid: 10283 timeout: 1995 lvb_type: 0 00000100:00100000:1.0:1769074907.670710:0:6514:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6514:1855007625269120:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074907.670784:0:6514:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5437ba000 going to sleep for 11 seconds 00000100:00100000:1.0:1769074907.680072:0:6514:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6514:1855007625269120:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074907.680105:0:6514:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00000100:00100000:0.0:1769074907.683550:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074907.683986:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617055104 00000100:00100000:2.0:1769074907.683998:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617055104 00010000:00010000:2.0:1769074907.684042:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 12 pid: 10283 timeout: 1995 lvb_type: 0 00000100:00100000:2.0:1769074907.684152:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7233 00000100:00100000:2.0:1769074907.684163:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617055104:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074907.684181:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074907.684187:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 12 pid: 10283 timeout: 1995 lvb_type: 0 00010000:00010000:2.0:1769074907.684202:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 12 pid: 10283 timeout: 1995 lvb_type: 0 00010000:00010000:2.0:1769074907.684224:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55d85 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c99e expref: 12 pid: 10283 timeout: 1995 lvb_type: 0 00010000:00010000:2.0:1769074907.684240:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074907.684246:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074907.684256:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074907.684263:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 4/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074907.684295:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074907.684364:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617055104:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 201us (822us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074907.684372:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7233 00010000:00010000:1.0:1769074907.684610:0:6514:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.684619:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00000004:00200000:1.0:1769074907.684643:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 0 00000004:00080000:1.0:1769074907.684849:0:6514:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x36:0x0], index 0: 0 00000004:00080000:1.0:1769074907.684858:0:6514:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x37:0x0], index 1: 0 00010000:00010000:1.0:1769074907.684982:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.684991:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 2/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.684997:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.685002:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074907.685006:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.685013:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 4/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.685026:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b64effa800) 00010000:00010000:1.0:1769074907.685029:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074907.685033:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 2/0,0 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.685038:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55d9a lrc: 0/0,0 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.685100:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55d93 lrc: 1/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9a5 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074907.685109:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b64effa800) 00010000:00010000:1.0:1769074907.685111:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074907.685114:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55d93 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9a5 expref: 11 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074907.685128:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242800, rc 0) 00000100:00100000:1.0:1769074907.685171:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68436:x1855007617054976:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 14704us (14904us total) trans 4294968374 rc 301/301 00000100:00100000:1.0:1769074907.685179:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13645 00000100:00100000:0.0:1769074907.695536:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074907.696138:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617055360 00000100:00100000:2.0:1769074907.696151:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617055360 00000100:00100000:2.0:1769074907.696260:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4748 00000100:00100000:2.0:1769074907.696271:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68436:x1855007617055360:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074907.696608:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68436:x1855007617055360:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 339us (1081us total) trans 4294968375 rc 0/0 00000100:00100000:2.0:1769074907.696616:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4748 00000100:00100000:3.0:1769074907.711943:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074907.712164:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617055616 00000100:00100000:2.0:1769074907.712176:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617055616 00000100:00100000:2.0:1769074907.712235:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4749 00000100:00100000:2.0:1769074907.712245:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68436:x1855007617055616:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074907.712509:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68436:x1855007617055616:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 257us (568us total) trans 4294968376 rc 0/0 00000100:00100000:2.0:1769074907.712516:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4749 00000100:00100000:1.0:1769074907.731846:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074907.732008:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617055872 00000100:00100000:0.0:1769074907.732020:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617055872 00000100:00100000:0.0:1769074907.732097:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13646 00000100:00100000:0.0:1769074907.732108:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68437:x1855007617055872:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074907.732132:0:6514:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074907.732155:0:6514:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd55da8 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9ac expref: -99 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.732236:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.732249:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732260:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732271:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074907.732278:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732284:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732365:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732373:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732378:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732384:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732398:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074907.732403:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732410:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732421:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b643a80400) 00010000:00010000:0.0:1769074907.732424:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074907.732430:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732436:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55daf lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732460:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732466:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732471:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732476:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074907.732479:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732484:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732491:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa000) 00010000:00010000:0.0:1769074907.732493:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074907.732497:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732502:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55db6 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074907.732529:0:6514:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd55da8 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9ac expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.732539:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b643a80400) 00010000:00010000:0.0:1769074907.732541:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1769074907.732544:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd55da8 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9ac expref: 13 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074907.732557:0:6514:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135c00, rc 0) 00000100:00100000:0.0:1769074907.732620:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68437:x1855007617055872:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 514us (826us total) trans 0 rc 301/301 00000100:00100000:0.0:1769074907.732628:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13646 00000100:00100000:3.0:1769074907.738154:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074907.738464:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617056128 00000100:00100000:2.0:1769074907.738476:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617056128 00000100:00100000:2.0:1769074907.738533:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13647 00000100:00100000:2.0:1769074907.738543:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:68437:x1855007617056128:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074907.738562:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074907.738583:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55dbd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9b3 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1769074907.738730:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20c:0x0]: returning size 4096 00010000:00010000:2.0:1769074907.738880:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55dbd lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9b3 expref: 27 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.738894:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074907.738897:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074907.738902:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd55dbd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9b3 expref: 27 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074907.738916:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88000, rc 0) 00000100:00100000:2.0:1769074907.738969:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:68437:x1855007617056128:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 428us (826us total) trans 8589944140 rc 301/301 00000100:00100000:2.0:1769074907.738977:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13647 00000100:00100000:3.0:1769074907.742554:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074907.742662:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617056256 00000100:00100000:0.0:1769074907.742672:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617056256 00000100:00100000:0.0:1769074907.742735:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13648 00000100:00100000:0.0:1769074907.742746:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68437:x1855007617056256:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:1.0:1769074907.743611:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074907.743631:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.743647:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.743662:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074907.743670:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.743676:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.743767:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074907.743776:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.743782:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00000004:00200000:1.0:1769074907.743991:0:6514:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 0 00000004:00200000:1.0:1769074907.743996:0:6514:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074907.744011:0:6514:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744027:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744033:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744039:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744057:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744063:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074907.744067:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744075:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744087:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b665615700) 00010000:00010000:1.0:1769074907.744090:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074907.744095:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744101:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd55dcb lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744122:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744129:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744135:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744139:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074907.744142:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744148:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744155:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa000) 00010000:00010000:1.0:1769074907.744157:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074907.744161:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744166:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243a00/0x7fe2323adfd55dd2 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744175:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744181:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744187:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744191:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074907.744194:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744199:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744205:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b665615d00) 00010000:00010000:1.0:1769074907.744208:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074907.744210:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074907.744215:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242e00/0x7fe2323adfd55dd9 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00000100:00100000:1.0:1769074907.744316:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543858e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68437:x1855007617056256:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1569us (1772us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074907.744326:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13648 00000100:00100000:1.0:1769074907.747906:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074907.748068:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617056512 00000100:00100000:2.0:1769074907.748081:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617056512 00000100:00100000:2.0:1769074907.748130:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4750 00000100:00100000:2.0:1769074907.748139:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:68437:x1855007617056512:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074907.748370:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:68437:x1855007617056512:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 230us (474us total) trans 8589944141 rc 0/0 00000100:00100000:2.0:1769074907.748377:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4750 00000100:00100000:1.0:1769074908.061735:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625269248:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074908.061851:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074908.062402:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625269248 00000100:00100000:3.0:1769074908.062411:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625269248 00000100:00100000:3.0:1769074908.062536:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1275 00000100:00100000:3.0:1769074908.062545:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625269248:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074908.062567:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074908.062624:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625269248:12345-0@lo:13:osp-pre-1-1.0 Request processed in 80us (773us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074908.062632:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1275 00000100:00100000:2.0:1769074908.062731:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625269248:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1769074908.062978:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1769074908.576748:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074909.598500:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074910.112731:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625269760:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.112847:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074910.112916:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625270272:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.112940:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074910.112956:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625270784:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.112976:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074910.113016:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1769074910.113024:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625269632:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.113048:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074910.113062:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625270144:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.113079:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074910.113094:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625270656:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.113115:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074910.113147:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625269760 00000100:00100000:3.0:1769074910.113157:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625269760 00000100:00100000:3.0:1769074910.113231:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270784 00000100:00100000:3.0:1769074910.113234:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270784 00000100:00100000:3.0:1769074910.113256:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270656 00000100:00100000:3.0:1769074910.113258:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270656 00000100:00100000:3.0:1769074910.113280:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1603 00000100:00100000:3.0:1769074910.113289:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625269760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113526:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270272 00000100:00100000:1.0:1769074910.113531:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270272 00000100:00100000:1.0:1769074910.113563:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625269632 00000100:00100000:1.0:1769074910.113566:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625269632 00000100:00100000:1.0:1769074910.113582:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270144 00000100:00100000:1.0:1769074910.113585:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270144 00000100:00100000:1.0:1769074910.113600:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2535 00000100:00100000:1.0:1769074910.113607:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625270272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113663:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625270272:12345-0@lo:400:kworker.0 Request processed in 57us (721us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074910.113671:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2535 00000100:00100000:1.0:1769074910.113689:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2536 00000100:00100000:1.0:1769074910.113695:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625269632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113721:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625269632:12345-0@lo:400:kworker.0 Request processed in 27us (672us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074910.113726:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2536 00000100:00100000:1.0:1769074910.113736:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2537 00000100:00100000:1.0:1769074910.113741:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625270144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113763:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625270144:12345-0@lo:400:kworker.0 Request processed in 23us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074910.113768:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2537 00000100:00100000:1.0:1769074910.113786:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1604 00000100:00100000:1.0:1769074910.113792:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625270784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113861:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625270784:12345-0@lo:400:kworker.0 Request processed in 69us (884us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074910.113867:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1604 00000100:00100000:1.0:1769074910.113881:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1605 00000100:00100000:1.0:1769074910.113887:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625270656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113911:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625270656:12345-0@lo:400:kworker.0 Request processed in 25us (795us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074910.113917:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1605 00000100:00100000:1.0:1769074910.113968:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625269632:0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113980:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625270144:0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113988:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625270656:0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.113997:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625270272:0@lo:400:kworker.0 00000100:00100000:1.0:1769074910.114020:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625270784:0@lo:400:kworker.0 00000100:00080000:1.0:1769074910.114071:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074910.114076:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074910.114079:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074910.114081:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074910.114785:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625269760:12345-0@lo:400:kworker.0 Request processed in 1493us (1934us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074910.114794:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1603 00000100:00100000:0.0:1769074910.114924:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625269504:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.114963:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.114997:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625270016:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115030:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.115050:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625270528:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115071:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.115095:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:0.0:1769074910.115102:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625269376:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115128:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.115151:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625269888:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115169:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.115183:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625270400:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115201:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.115218:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625270912:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115238:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074910.115266:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625269504 00000100:00100000:0.0:1769074910.115273:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625269504 00000100:00100000:0.0:1769074910.115321:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270528 00000100:00100000:0.0:1769074910.115325:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270528 00000100:00100000:0.0:1769074910.115342:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2771 00000100:00100000:0.0:1769074910.115348:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625269504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115386:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625269504:12345-0@lo:400:kworker.0 Request processed in 38us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074910.115391:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2771 00000100:00100000:0.0:1769074910.115404:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2772 00000100:00100000:0.0:1769074910.115409:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625270528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115434:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625270528:12345-0@lo:400:kworker.0 Request processed in 25us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074910.115440:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2772 00000100:00100000:0.0:1769074910.115456:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270016 00000100:00100000:0.0:1769074910.115471:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270016 00000100:00100000:0.0:1769074910.115503:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270400 00000100:00100000:0.0:1769074910.115507:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270400 00000100:00100000:0.0:1769074910.115519:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625270912 00000100:00100000:0.0:1769074910.115521:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625270912 00000100:00100000:0.0:1769074910.115531:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2538 00000100:00100000:0.0:1769074910.115536:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625270016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115565:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625270016:12345-0@lo:400:kworker.0 Request processed in 28us (534us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074910.115571:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2538 00000100:00100000:0.0:1769074910.115581:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2539 00000100:00100000:0.0:1769074910.115586:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625270400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115611:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625270400:12345-0@lo:400:kworker.0 Request processed in 26us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074910.115616:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2539 00000100:00100000:0.0:1769074910.115626:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2540 00000100:00100000:0.0:1769074910.115631:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625270912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115653:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625270912:12345-0@lo:400:kworker.0 Request processed in 22us (414us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074910.115659:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2540 00000100:00080000:0.0:1769074910.115681:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074910.115684:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074910.115718:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625270400:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115728:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625270912:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115736:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625269504:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115743:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625270016:0@lo:400:kworker.0 00000100:00100000:0.0:1769074910.115752:0:3650:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968372 gen 2 00000004:00080000:0.0:1769074910.115758:0:3650:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968372 committed transno 4294968372 00000100:00100000:0.0:1769074910.115782:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625270528:0@lo:400:kworker.0 00000100:00100000:2.0:1769074910.116260:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625269888 00000100:00100000:2.0:1769074910.116268:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625269888 00000100:00100000:2.0:1769074910.116338:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1606 00000100:00100000:2.0:1769074910.116345:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625269888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074910.116394:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625269888:12345-0@lo:400:kworker.0 Request processed in 49us (1222us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074910.116399:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1606 00000100:00100000:2.0:1769074910.116436:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625269888:0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.116785:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625269376 00000100:00100000:3.0:1769074910.116790:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625269376 00000100:00100000:3.0:1769074910.116858:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1769074910.116866:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625269376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074910.117232:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625269376:12345-0@lo:400:kworker.0 Request processed in 367us (2102us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074910.117239:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1769074910.117291:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625269376:0@lo:400:kworker.0 00000100:00080000:0.0:1769074910.117758:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074910.119102:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54385aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625269760:0@lo:400:kworker.0 00010000:00010000:0.0:1769074910.621262:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074911.171121:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074911.171169:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074911.171209:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074911.171241:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074911.171267:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074911.171640:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617056896 00000100:00100000:0.0:1769074911.171650:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617056896 00000100:00100000:0.0:1769074911.171697:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617056768 00000100:00100000:0.0:1769074911.171699:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617056768 00000100:00100000:0.0:1769074911.171716:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2541 00000100:00100000:0.0:1769074911.171722:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617056896:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074911.171782:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617056896:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 61us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074911.171788:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2541 00000100:00100000:0.0:1769074911.171799:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2542 00000100:00100000:0.0:1769074911.171803:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007617056768:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074911.171866:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007617056768:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 62us (624us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074911.171873:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2542 00000100:00080000:0.0:1769074911.171896:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074911.171900:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074911.172480:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057152 00000100:00100000:2.0:1769074911.172488:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057152 00000100:00100000:2.0:1769074911.172532:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057024 00000100:00100000:2.0:1769074911.172535:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057024 00000100:00100000:2.0:1769074911.172549:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1607 00000100:00100000:2.0:1769074911.172555:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617057152:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074911.172583:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617057152:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (1375us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074911.172587:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1607 00000100:00100000:2.0:1769074911.172596:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1608 00000100:00100000:2.0:1769074911.172600:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617057024:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074911.172611:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617057024:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 12us (1345us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074911.172615:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1608 00000100:00100000:2.0:1769074911.172629:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617056640 00000100:00100000:2.0:1769074911.172631:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617056640 00000100:00100000:2.0:1769074911.172645:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 397 00000100:00100000:2.0:1769074911.172649:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617056640:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074911.172663:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617056640:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 14us (1554us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074911.172667:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 397 00000100:00080000:0.0:1769074911.174128:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074911.174138:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1769074911.646205:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074912.157414:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625271040:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074912.157512:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074912.158594:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271040 00000100:00100000:1.0:1769074912.158605:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271040 00000100:00100000:1.0:1769074912.158669:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2773 00000100:00100000:1.0:1769074912.158678:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625271040:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074912.158710:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:1.0:1769074912.158776:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625271040:12345-0@lo:41:osp-pre-1-0.0 Request processed in 99us (1263us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074912.158783:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2773 00000100:00100000:2.0:1769074912.159029:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625271040:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074912.159286:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074912.159299:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625271168:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074912.159339:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074912.159656:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271168 00000100:00100000:3.0:1769074912.159663:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271168 00000100:00100000:3.0:1769074912.159707:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2774 00000100:00100000:3.0:1769074912.159716:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625271168:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074912.159738:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:3.0:1769074912.159783:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625271168:12345-0@lo:41:osp-pre-0-1.0 Request processed in 68us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074912.159790:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2774 00000100:00100000:2.0:1769074912.160494:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625271168:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074912.669929:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625271296:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074912.670026:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074912.670108:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271296 00000100:00100000:3.0:1769074912.670116:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271296 00000100:00100000:3.0:1769074912.670170:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1276 00000100:00100000:3.0:1769074912.670179:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625271296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074912.670206:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074912.670247:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625271296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074912.670254:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1276 00000100:00100000:3.0:1769074912.670316:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625271296:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1769074912.670475:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074912.670795:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074912.670825:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625271424:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074912.670867:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074912.670941:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271424 00000100:00100000:1.0:1769074912.670948:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271424 00000100:00100000:1.0:1769074912.670994:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1277 00000100:00100000:1.0:1769074912.671002:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625271424:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074912.671019:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074912.671058:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625271424:12345-0@lo:13:osp-pre-1-0.0 Request processed in 57us (190us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074912.671064:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1277 00000100:00100000:1.0:1769074912.671096:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625271424:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1769074912.672476:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1769074912.672489:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074912.673051:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625271552:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074912.673109:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074912.673215:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271552 00000100:00100000:1.0:1769074912.673223:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271552 00000100:00100000:1.0:1769074912.673293:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1278 00000100:00100000:1.0:1769074912.673301:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625271552:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074912.673321:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074912.673366:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625271552:12345-0@lo:13:osp-pre-0-1.0 Request processed in 65us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074912.673373:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1278 00000100:00080000:0.0:1769074912.675312:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074912.675399:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54cf69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625271552:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074913.181320:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625271680:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074913.181412:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074913.181800:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271680 00000100:00100000:3.0:1769074913.181963:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271680 00000100:00100000:3.0:1769074913.182042:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1279 00000100:00100000:3.0:1769074913.182053:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625271680:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074913.182074:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074913.182123:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625271680:12345-0@lo:13:osp-pre-1-1.0 Request processed in 73us (711us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074913.182130:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1279 00000100:00080000:1.0:1769074913.182410:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074913.182463:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625271680:0@lo:13:osp-pre-1-1.0 00000001:00080000:1.0:1769074913.188596:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968376 is committed 00010000:00010000:0.0:1769074913.190614:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074913.190633:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 2/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074913.190641:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074913.190647:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074913.190652:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074913.190660:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 4/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074913.190682:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa000) 00010000:00010000:0.0:1769074913.190685:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074913.190690:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 2/0,0 mode: --/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074913.190696:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89e00/0x7fe2323adfd55d7e lrc: 0/0,0 mode: --/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000001:00080000:3.0:1769074913.193359:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944141 is committed 00000020:00080000:3.0:1769074913.193379:0:32377:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff89b67fc18d80 refcount 2 committed 0 result 0 batchid 4294969369 00000020:00080000:3.0:1769074913.193385:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b542f2e800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff89b67fc18480 00000020:00080000:3.0:1769074913.193388:0:32377:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000405:0x2:0x0].9 00000020:00080000:3.0:1769074913.193391:0:32377:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff89b64bb5a200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff89b54cc68600 00000020:00080000:3.0:1769074913.193394:0:32377:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].87 00000020:00080000:2.0:1769074913.194139:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294969369 committed batchid 4294969368 00000020:00080000:2.0:1769074913.194221:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969368 00010000:00010000:0.0:1769074913.194519:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194531:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194539:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194546:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074913.194551:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194559:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194578:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8f26d1c0 (ffff89b67a317000) 00010000:00010000:0.0:1769074913.194581:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074913.194586:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194591:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd55ce4 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x8f26d1c0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194614:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194620:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 2/0,0 mode: TXN/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194625:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194630:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67feba800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074913.194632:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 3/0,0 mode: TXN/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194639:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 4/0,0 mode: TXN/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194646:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20c:0x0].0x0 (ffff89b64a19d300) 00010000:00010000:0.0:1769074913.194648:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074913.194651:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 2/0,0 mode: --/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194656:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67feba800/0x7fe2323adfd55ceb lrc: 0/0,0 mode: --/TXN res: [0x200000405:0x20c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194665:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194671:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194677:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194682:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074913.194685:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194690:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194710:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xa:0x0].0x0 (ffff89b64a19dc00) 00010000:00010000:0.0:1769074913.194712:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074913.194715:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 2/0,0 mode: --/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.194721:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55cf2 lrc: 0/0,0 mode: --/TXN res: [0x200000bd0:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT0 00010000:00010000:0.0:1769074913.693280:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074914.717072:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074915.229305:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625272064:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229384:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074915.229449:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625272576:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229470:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074915.229488:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625273088:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229508:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074915.229548:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1769074915.229554:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625271936:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229576:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074915.229595:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625272448:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229612:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074915.229628:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625272960:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229644:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074915.229676:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272064 00000100:00100000:0.0:1769074915.229684:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272064 00000100:00100000:0.0:1769074915.229727:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272576 00000100:00100000:0.0:1769074915.229730:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272576 00000100:00100000:0.0:1769074915.229754:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272448 00000100:00100000:0.0:1769074915.229757:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272448 00000100:00100000:0.0:1769074915.229773:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2543 00000100:00100000:0.0:1769074915.229780:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625272064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229858:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625272064:12345-0@lo:400:kworker.0 Request processed in 78us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074915.229865:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2543 00000100:00100000:0.0:1769074915.229881:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2544 00000100:00100000:0.0:1769074915.229886:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625272576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229912:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625272576:12345-0@lo:400:kworker.0 Request processed in 26us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074915.229930:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2544 00000100:00100000:0.0:1769074915.229941:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2545 00000100:00100000:0.0:1769074915.229947:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625272448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.229971:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625272448:12345-0@lo:400:kworker.0 Request processed in 25us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074915.229977:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2545 00000100:00100000:0.0:1769074915.229997:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271936 00000100:00100000:0.0:1769074915.230000:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271936 00000100:00100000:0.0:1769074915.230020:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272960 00000100:00100000:0.0:1769074915.230025:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272960 00000100:00100000:0.0:1769074915.230040:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2775 00000100:00100000:0.0:1769074915.230045:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625271936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.230074:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625271936:12345-0@lo:400:kworker.0 Request processed in 28us (496us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074915.230079:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2775 00000100:00100000:0.0:1769074915.230091:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2776 00000100:00100000:0.0:1769074915.230097:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625272960:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.230119:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625272960:12345-0@lo:400:kworker.0 Request processed in 22us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074915.230123:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2776 00000100:00100000:0.0:1769074915.230171:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625271936:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.230183:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625272448:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.230192:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625272960:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.230200:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625272064:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.230207:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625272576:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230491:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625272192:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230523:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074915.230559:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625272704:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230582:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074915.230599:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625273216:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230620:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074915.230638:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:1.0:1769074915.230643:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625271808:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230667:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074915.230687:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625272320:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230708:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074915.230725:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625272832:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230744:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074915.230756:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625273344:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230772:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1769074915.230794:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074915.230798:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074915.230833:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273088 00000100:00100000:1.0:1769074915.230839:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273088 00000100:00100000:1.0:1769074915.230872:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272192 00000100:00100000:1.0:1769074915.230875:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272192 00000100:00100000:1.0:1769074915.230892:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273216 00000100:00100000:1.0:1769074915.230894:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273216 00000100:00100000:1.0:1769074915.230910:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272320 00000100:00100000:1.0:1769074915.230913:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272320 00000100:00100000:1.0:1769074915.230943:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1609 00000100:00100000:1.0:1769074915.230949:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625273088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.230987:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625273088:12345-0@lo:400:kworker.0 Request processed in 39us (1478us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074915.230993:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1609 00000100:00100000:1.0:1769074915.231007:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1610 00000100:00100000:1.0:1769074915.231012:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625272192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.231045:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625272192:12345-0@lo:400:kworker.0 Request processed in 34us (521us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074915.231050:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1610 00000100:00100000:1.0:1769074915.231060:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1611 00000100:00100000:1.0:1769074915.231065:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625273216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.231089:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625273216:12345-0@lo:400:kworker.0 Request processed in 24us (469us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074915.231094:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1611 00000100:00100000:1.0:1769074915.231103:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1612 00000100:00100000:1.0:1769074915.231108:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625272320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.231131:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625272320:12345-0@lo:400:kworker.0 Request processed in 22us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074915.231136:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1612 00000100:00100000:1.0:1769074915.231162:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272704 00000100:00100000:1.0:1769074915.231166:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272704 00000100:00100000:1.0:1769074915.231185:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625272832 00000100:00100000:1.0:1769074915.231187:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625272832 00000100:00100000:1.0:1769074915.231198:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273344 00000100:00100000:1.0:1769074915.231200:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273344 00000100:00100000:1.0:1769074915.231211:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2546 00000100:00100000:1.0:1769074915.231216:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625272704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.231241:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625272704:12345-0@lo:400:kworker.0 Request processed in 25us (659us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074915.231246:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2546 00000100:00100000:1.0:1769074915.231256:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2547 00000100:00100000:1.0:1769074915.231262:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625272832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.231484:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2548 00000100:00100000:0.0:1769074915.231497:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625273344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.231530:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625273344:12345-0@lo:400:kworker.0 Request processed in 37us (757us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074915.231536:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2548 00000100:00100000:0.0:1769074915.231577:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625273088:0@lo:400:kworker.0 00000100:00080000:0.0:1769074915.231608:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074915.231612:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074915.231615:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074915.231617:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074915.231638:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625272320:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.231648:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625273344:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.231657:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625272192:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.231666:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625272704:0@lo:400:kworker.0 00000100:00100000:0.0:1769074915.231675:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625273216:0@lo:400:kworker.0 00000100:00100000:3.0:1769074915.231775:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625271808 00000100:00100000:3.0:1769074915.231781:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625271808 00000100:00100000:3.0:1769074915.231836:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1769074915.231843:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625271808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074915.231882:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625271808:12345-0@lo:400:kworker.0 Request processed in 40us (1214us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074915.231889:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1769074915.231941:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625271808:0@lo:400:kworker.0 00000100:00100000:1.0:1769074915.232046:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625272832:12345-0@lo:400:kworker.0 Request processed in 784us (1300us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074915.232052:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2547 00000100:00100000:3.0:1769074915.232512:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625272832:0@lo:400:kworker.0 00010000:00010000:0.0:1769074915.742762:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074916.295365:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074916.295475:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057280 00000100:00100000:0.0:1769074916.295487:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057280 00000100:00100000:0.0:1769074916.295539:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 399 00000100:00100000:0.0:1769074916.295548:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617057280:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074916.295620:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617057280:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 73us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074916.295629:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 399 00000100:00100000:1.0:1769074916.300252:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074916.300313:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074916.300411:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057408 00000100:00100000:0.0:1769074916.300421:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057408 00000100:00100000:0.0:1769074916.300507:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2549 00000100:00100000:0.0:1769074916.300517:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007617057408:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074916.300590:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2409:x1855007617057408:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 74us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074916.300598:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2549 00000100:00100000:3.0:1769074916.300875:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057664 00000100:00100000:3.0:1769074916.300880:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057664 00000100:00100000:3.0:1769074916.300924:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1613 00000100:00100000:3.0:1769074916.300933:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617057664:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074916.300974:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617057664:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 42us (660us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074916.300981:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1613 00000100:00080000:1.0:1769074916.301691:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074916.301697:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074916.313081:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074916.313160:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074916.313244:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057536 00000100:00100000:2.0:1769074916.313264:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057536 00000100:00100000:2.0:1769074916.313343:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2550 00000100:00100000:2.0:1769074916.313354:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617057536:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074916.313433:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617057536:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 82us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074916.313441:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2550 00000100:00100000:0.0:1769074916.313893:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617057792 00000100:00100000:0.0:1769074916.313909:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617057792 00000100:00100000:0.0:1769074916.313945:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1614 00000100:00100000:0.0:1769074916.313953:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54385ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617057792:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074916.314005:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54385ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2408:x1855007617057792:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 53us (845us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074916.314012:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1614 00000100:00080000:1.0:1769074916.314664:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074916.314672:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:0.0:1769074916.765355:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074917.277163:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074917.277186:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625273472:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074917.277251:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074917.277349:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273472 00000100:00100000:0.0:1769074917.277357:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273472 00000100:00100000:0.0:1769074917.277408:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2777 00000100:00100000:0.0:1769074917.277418:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625273472:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074917.277452:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:0.0:1769074917.277501:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625273472:12345-0@lo:41:osp-pre-0-1.0 Request processed in 85us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074917.277511:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2777 00000100:00100000:0.0:1769074917.277556:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625273472:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074917.278718:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074917.278744:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625273600:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074917.278793:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074917.278975:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273600 00000100:00100000:2.0:1769074917.278983:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273600 00000100:00100000:2.0:1769074917.279033:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2778 00000100:00100000:2.0:1769074917.279042:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625273600:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074917.279069:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:2.0:1769074917.279120:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625273600:12345-0@lo:41:osp-pre-1-0.0 Request processed in 79us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074917.279126:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2778 00000100:00100000:3.0:1769074917.279205:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543859880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625273600:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074917.789058:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074917.789080:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625273728:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074917.789157:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074917.789227:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625273856:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074917.789249:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074917.789249:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273728 00000100:00100000:1.0:1769074917.789256:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273728 00000100:00080000:0.0:1769074917.789279:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074917.789321:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273856 00000100:00100000:1.0:1769074917.789324:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273856 00000100:00100000:3.0:1769074917.789345:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1280 00000100:00100000:3.0:1769074917.789356:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625273728:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074917.789358:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1281 00000100:00100000:1.0:1769074917.789364:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625273856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074917.789386:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00002000:00000024:3.0:1769074917.789386:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074917.789435:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625273856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (185us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074917.789442:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1281 00000100:00080000:1.0:1769074917.789464:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074917.789547:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625273984:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1769074917.789557:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074917.789573:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074917.789601:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625273984 00000100:00100000:0.0:1769074917.789606:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625273984 00000100:00100000:0.0:1769074917.789643:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1282 00000100:00100000:0.0:1769074917.789649:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625273984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074917.789661:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:0.0:1769074917.789690:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625273984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (117us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074917.789695:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1282 00000100:00100000:0.0:1769074917.789732:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549200700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625273984:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1769074917.789966:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074917.790225:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625273728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 870us (1066us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074917.790232:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1280 00000100:00100000:1.0:1769074917.790933:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625273856:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074917.790954:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625273728:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074918.301237:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625274112:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074918.301316:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074918.301440:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625274112 00000100:00100000:0.0:1769074918.301450:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625274112 00000100:00100000:0.0:1769074918.301531:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1283 00000100:00100000:0.0:1769074918.301541:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625274112:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074918.301563:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00080000:1.0:1769074918.301988:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074918.308033:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549202680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625274112:12345-0@lo:13:osp-pre-1-1.0 Request processed in 6482us (6706us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074918.308050:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1283 00000100:00100000:2.0:1769074918.315320:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625274112:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1769074918.813304:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1769074919.328876:0:32377:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 4294969369 batchid updated 00000020:00080000:2.0:1769074919.328981:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969369 00000004:00080000:2.0:1769074919.329047:0:32406:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b681689a80 with version 24 00000020:00080000:2.0:1769074919.329066:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294969369 cancel update log [0x240000405:0x2:0x0].9: rc = 0 00000004:00080000:1.0:1769074919.329122:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 24 rpc_version 24 00000020:00080000:2.0:1769074919.329141:0:32406:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294969369 cancel update log [0x200000bd0:0x2:0x0].87: rc = 0 00000020:00080000:2.0:1769074919.329151:0:32406:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294969369 00000020:00100000:1.0:1769074919.329152:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1769074919.329163:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b543858a80 x1855007625274368/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1769074919.329187:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b543858a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625274368:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074919.329250:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074919.329334:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5437bab40 going to sleep for 11 seconds 00000100:00100000:0.0:1769074919.335781:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625274368 00000100:00100000:0.0:1769074919.335799:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625274368 00000100:00100000:0.0:1769074919.335958:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2779 00000100:00100000:0.0:1769074919.335968:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b543859880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625274368:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1769074919.336198:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b543859880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625274368:12345-0@lo:1000:osp_up1-0.0 Request processed in 231us (6947us total) trans 4294968377 rc 0/0 00000100:00100000:0.0:1769074919.336206:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2779 00000100:00100000:1.0:1769074919.336753:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b543858a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625274368:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074919.624128:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074919.624228:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617058048 00000100:00100000:3.0:1769074919.624238:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617058048 00000100:00100000:3.0:1769074919.624321:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13649 00000100:00100000:3.0:1769074919.624332:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:68465:x1855007617058048:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074919.624351:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074919.624375:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55de0 lrc: 2/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9ba expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074919.624522:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074919.624539:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624550:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624562:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074919.624568:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624573:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624631:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55df5 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074919.624638:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55df5 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624645:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55df5 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00000004:00200000:3.0:1769074919.624654:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 0 00010000:00010000:3.0:1769074919.624732:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624739:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624745:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624752:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624758:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074919.624762:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624769:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624781:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074919.624784:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074919.624788:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624794:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55de7 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624840:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624855:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624860:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624865:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbe00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074919.624868:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624875:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624882:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea100) 00010000:00010000:3.0:1769074919.624884:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074919.624887:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624893:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55dee lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.624931:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55de0 lrc: 0/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9ba expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074919.624945:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55df5 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9ba expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074919.624952:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543eae00) 00010000:00010000:3.0:1769074919.624954:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074919.624956:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074919.625026:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68465:x1855007617058048:12345-192.168.204.60@tcp:101:dd.0 Request processed in 696us (908us total) trans 4294968378 rc 0/0 00000100:00100000:3.0:1769074919.625033:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13649 00000100:00080000:1.0:1769074919.626083:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074919.656405:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074919.657096:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617058304 00000100:00100000:3.0:1769074919.657107:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617058304 00000100:00100000:3.0:1769074919.657170:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13650 00000100:00100000:3.0:1769074919.657179:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68465:x1855007617058304:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:3.0:1769074919.657205:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074919.657209:0:6513:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769074919.657236:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55df5 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9ba expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074919.657256:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543eae00) 00010000:00010000:3.0:1769074919.657266:0:6513:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074919.657291:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e03 lrc: 2/0,0 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9c1 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074919.657335:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55e0a lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074919.657347:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55e0a lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074919.657356:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55e0a lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00000004:00080000:3.0:1769074919.657590:0:6513:0:(osp_object.c:1649:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x38:0x0], index 1: 0 00000040:00080000:3.0:1769074919.657670:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].67, 88 off13536 00000004:00080000:3.0:1769074919.657680:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1907499872365ns, next at 1833854042001ns, rc = 0 00000040:00080000:3.0:1769074919.657723:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].71, 88 off13792 00000004:00080000:3.0:1769074919.657729:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1907499921896ns, next at 1835465451293ns, rc = 0 00000040:00080000:3.0:1769074919.657759:0:6513:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].72, 88 off13880 00000004:00080000:3.0:1769074919.657764:0:6513:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1907499956190ns, next at 1835465451293ns, rc = 0 00000001:00080000:1.0:1769074919.660481:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968379 is committed 00000004:00080000:1.0:1769074919.660500:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 71 committed 00000004:00080000:1.0:1769074919.660524:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 70 committed 00000004:00080000:1.0:1769074919.660527:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 66 committed 00000001:00080000:1.0:1769074919.660549:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968377 is committed 00000100:00100000:0.0:1769074919.660771:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625274624:0@lo:2:osp-syn-1-1.0 00010000:00010000:3.0:1769074919.660804:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55e0a lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c1 expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074919.660861:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074919.660864:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074919.660882:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e03 lrc: 0/0,0 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c1 expref: 9 pid: 6513 timeout: 0 lvb_type: 3 00010000:00010000:3.0:1769074919.660903:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55e0a lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0xa/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c1 expref: 8 pid: 6513 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074919.660916:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1769074919.660949:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074919.660955:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1769074919.660958:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbe00, rc 0) 00000100:00100000:3.0:1769074919.661050:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68465:x1855007617058304:12345-192.168.204.60@tcp:101:dd.0 Request processed in 3867us (4651us total) trans 4294968379 rc 0/0 00000100:00100000:3.0:1769074919.661061:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13650 00000100:00100000:1.0:1769074919.661072:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625274496:0@lo:2:osp-syn-0-1.0 00000100:00100000:1.0:1769074919.661114:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074919.661215:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625274624 00000100:00100000:1.0:1769074919.661226:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625274624 00000100:00100000:3.0:1769074919.661256:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074919.661284:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625274496 00000100:00100000:1.0:1769074919.661289:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625274496 00000100:00100000:3.0:1769074919.661290:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625274752:0@lo:2:osp-syn-1-1.0 00000100:00100000:1.0:1769074919.661334:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14542 00000100:00100000:1.0:1769074919.661346:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625274624:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:3.0:1769074919.661361:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074919.661413:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625274752 00000100:00100000:3.0:1769074919.661421:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625274752 00000100:00100000:3.0:1769074919.661476:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14543 00000100:00100000:3.0:1769074919.661486:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625274496:12345-0@lo:2:osp-syn-0-1.0 00000100:00100000:3.0:1769074919.661677:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625274496:12345-0@lo:2:osp-syn-0-1.0 Request processed in 193us (562us total) trans 21474841313 rc 0/0 00000100:00100000:3.0:1769074919.661683:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14543 00000100:00100000:3.0:1769074919.661698:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14544 00000100:00100000:3.0:1769074919.661704:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625274752:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:3.0:1769074919.661873:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+8:3650:x1855007625274752:12345-0@lo:2:osp-syn-1-1.0 Request processed in 168us (522us total) trans 17179874646 rc 0/0 00000100:00100000:3.0:1769074919.661879:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14544 00000004:00080000:3.0:1769074919.661931:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549f29500/2, rc 0, transno 21474841313 00000100:00100000:3.0:1769074919.661937:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625274496:0@lo:2:osp-syn-0-1.0 00000004:00080000:3.0:1769074919.661955:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b549208380/2, rc 0, transno 17179874646 00000100:00100000:3.0:1769074919.661960:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549208380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625274752:0@lo:2:osp-syn-1-1.0 00000100:00080000:3.0:1769074919.662009:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074919.673860:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625274624:12345-0@lo:2:osp-syn-1-1.0 Request processed in 12494us (12929us total) trans 17179874645 rc 0/0 00000100:00100000:1.0:1769074919.673877:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14542 00000100:00100000:3.0:1769074919.673942:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074919.674025:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617058432 00000100:00100000:1.0:1769074919.674036:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617058432 00000100:00100000:1.0:1769074919.674104:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14545 00000100:00100000:1.0:1769074919.674115:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:68465:x1855007617058432:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:1.0:1769074919.674132:0:28519:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074919.674193:0:28519:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9c8 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074919.674236:0:28519:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x38:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074919.674259:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074919.674262:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074919.674271:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 8 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074919.674288:0:28519:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 8 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074919.674301:0:28519:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242000, rc 0) 00000100:00080000:3.0:1769074919.674430:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074919.674713:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68465:x1855007617058432:12345-192.168.204.60@tcp:101:dd.0 Request processed in 598us (772us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074919.674721:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14545 00000004:00080000:0.0:1769074919.675722:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54920a680/2, rc 0, transno 17179874645 00000100:00100000:0.0:1769074919.675741:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625274624:0@lo:2:osp-syn-1-1.0 00000100:00100000:3.0:1769074919.685418:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074919.685748:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617058560 00000100:00100000:0.0:1769074919.685770:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617058560 00000100:00100000:0.0:1769074919.685904:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13651 00000100:00100000:0.0:1769074919.685918:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68465:x1855007617058560:12345-192.168.204.60@tcp:101:dd.0 00010000:00010000:0.0:1769074919.685957:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074919.685986:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55e18 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9cf expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074919.686033:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55e1f lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:0.0:1769074919.686049:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55e1f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:0.0:1769074919.686067:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55e1f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:0.0:1769074919.686226:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135e00/0x7fe2323adfd55e18 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9cf expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074919.686249:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55e1f lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9cf expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074919.686259:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074919.686262:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1769074919.686267:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135000, rc 0) 00000100:00100000:0.0:1769074919.686368:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68465:x1855007617058560:12345-192.168.204.60@tcp:101:dd.0 Request processed in 452us (960us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074919.686377:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13651 00000100:00100000:2.0:1769074919.762957:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074919.763213:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617058816 00000100:00100000:0.0:1769074919.763232:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617058816 00000100:00100000:0.0:1769074919.763327:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4751 00000100:00100000:0.0:1769074919.763339:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68465:x1855007617058816:12345-192.168.204.60@tcp:35:dd.0 00000100:00100000:0.0:1769074919.763888:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68465:x1855007617058816:12345-192.168.204.60@tcp:35:dd.0 Request processed in 550us (942us total) trans 4294968380 rc 0/0 00000100:00100000:0.0:1769074919.763899:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4751 00010000:00010000:0.0:1769074919.839472:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074920.360586:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625275264:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360662:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.360714:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625275776:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360738:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.360754:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625276288:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360773:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.360797:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:0.0:1769074920.360803:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625274880:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360862:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.360885:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625275392:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360905:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.360921:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625275904:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360940:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.360969:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625276416:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.360989:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361014:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549209180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625275008:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361040:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361058:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625275520:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361077:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361092:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625276032:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361115:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361131:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1769074920.361136:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545388e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625275136:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361156:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361172:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625275648:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361192:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361202:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625276160:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361219:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074920.361251:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275264 00000100:00100000:0.0:1769074920.361411:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275264 00000100:00100000:0.0:1769074920.361460:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276288 00000100:00100000:0.0:1769074920.361464:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276288 00000100:00100000:0.0:1769074920.361493:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275392 00000100:00100000:0.0:1769074920.361496:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275392 00000100:00100000:0.0:1769074920.361513:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276160 00000100:00100000:0.0:1769074920.361515:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276160 00000100:00100000:0.0:1769074920.361535:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1615 00000100:00100000:0.0:1769074920.361543:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3649:x1855007625275264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361598:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3649:x1855007625275264:12345-0@lo:400:kworker.0 Request processed in 56us (935us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.361605:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1615 00000100:00100000:0.0:1769074920.361623:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1616 00000100:00100000:0.0:1769074920.361629:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625276288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361656:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625276288:12345-0@lo:400:kworker.0 Request processed in 28us (882us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.361662:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1616 00000100:00100000:0.0:1769074920.361673:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1617 00000100:00100000:0.0:1769074920.361679:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625275392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361704:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625275392:12345-0@lo:400:kworker.0 Request processed in 25us (798us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.361709:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1617 00000100:00100000:0.0:1769074920.361721:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1618 00000100:00100000:0.0:1769074920.361726:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625276160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361755:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625276160:12345-0@lo:400:kworker.0 Request processed in 29us (535us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.361761:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1618 00000100:00100000:0.0:1769074920.361779:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275776 00000100:00100000:0.0:1769074920.361782:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275776 00000100:00100000:0.0:1769074920.361800:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275904 00000100:00100000:0.0:1769074920.361803:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275904 00000100:00100000:0.0:1769074920.361853:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276416 00000100:00100000:0.0:1769074920.361856:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276416 00000100:00100000:0.0:1769074920.361867:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275520 00000100:00100000:0.0:1769074920.361869:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275520 00000100:00100000:0.0:1769074920.361880:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275136 00000100:00100000:0.0:1769074920.361883:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275136 00000100:00100000:0.0:1769074920.361892:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275648 00000100:00100000:0.0:1769074920.361894:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275648 00000100:00100000:0.0:1769074920.361905:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2551 00000100:00100000:0.0:1769074920.361910:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625275776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.361942:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625275776:12345-0@lo:400:kworker.0 Request processed in 30us (1203us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.361948:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2551 00000100:00100000:0.0:1769074920.362117:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2552 00000100:00100000:0.0:1769074920.362125:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625275904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362155:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625275904:12345-0@lo:400:kworker.0 Request processed in 32us (1214us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362160:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2552 00000100:00100000:0.0:1769074920.362171:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2553 00000100:00100000:0.0:1769074920.362176:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625276416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362204:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625276416:12345-0@lo:400:kworker.0 Request processed in 27us (1213us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362209:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2553 00000100:00100000:0.0:1769074920.362220:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2554 00000100:00100000:0.0:1769074920.362225:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625275520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362248:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625275520:12345-0@lo:400:kworker.0 Request processed in 23us (1170us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362253:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2554 00000100:00100000:0.0:1769074920.362347:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2555 00000100:00100000:0.0:1769074920.362353:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625275136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362383:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625275136:12345-0@lo:400:kworker.0 Request processed in 31us (1226us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362388:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2555 00000100:00100000:0.0:1769074920.362399:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2556 00000100:00100000:0.0:1769074920.362405:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625275648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362429:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625275648:12345-0@lo:400:kworker.0 Request processed in 24us (1236us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362434:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2556 00000100:00100000:0.0:1769074920.362461:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625274880 00000100:00100000:0.0:1769074920.362466:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625274880 00000100:00100000:0.0:1769074920.362493:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1769074920.362499:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b08c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625274880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362528:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b08c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625274880:12345-0@lo:400:kworker.0 Request processed in 30us (1666us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362534:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1769074920.362574:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625275008 00000100:00100000:0.0:1769074920.362578:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625275008 00000100:00100000:0.0:1769074920.362600:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276032 00000100:00100000:0.0:1769074920.362602:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276032 00000100:00100000:0.0:1769074920.362618:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2780 00000100:00100000:0.0:1769074920.362623:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625275008:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362655:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625275008:12345-0@lo:400:kworker.0 Request processed in 30us (1613us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362660:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2780 00000100:00100000:0.0:1769074920.362671:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2781 00000100:00100000:0.0:1769074920.362677:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625276032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.362706:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b542fabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625276032:12345-0@lo:400:kworker.0 Request processed in 30us (1591us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074920.362711:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2781 00000100:00080000:0.0:1769074920.362742:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074920.362747:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074920.362749:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074920.362752:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074920.362791:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625274880:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364320:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625275392:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364356:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625275904:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364370:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625276416:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364386:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545389f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625275264:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364401:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625275776:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364417:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625276288:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364493:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545388e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625275136:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364508:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625275648:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364523:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625276160:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364541:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549209180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625275008:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364555:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625275520:0@lo:400:kworker.0 00000100:00100000:0.0:1769074920.364566:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968377 gen 2 00000004:00080000:0.0:1769074920.364572:0:3651:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968377 committed transno 4294968377 00000100:00100000:0.0:1769074920.364589:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625276032:0@lo:400:kworker.0 00010000:00010000:2.0:1769074920.861997:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074921.411032:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074921.411066:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074921.411091:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074921.411106:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074921.411130:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074921.411171:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617058944 00000100:00100000:3.0:1769074921.411180:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617058944 00000100:00100000:3.0:1769074921.411233:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 401 00000100:00100000:3.0:1769074921.411244:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617058944:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074921.411315:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617058944:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 73us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074921.411321:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 401 00000100:00100000:3.0:1769074921.411353:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059072 00000100:00100000:3.0:1769074921.411358:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059072 00000100:00100000:3.0:1769074921.411391:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059200 00000100:00100000:3.0:1769074921.411394:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059200 00000100:00100000:3.0:1769074921.411416:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2557 00000100:00100000:3.0:1769074921.411422:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007617059072:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074921.411453:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2406:x1855007617059072:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 31us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074921.411460:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2557 00000100:00100000:3.0:1769074921.411472:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2558 00000100:00100000:3.0:1769074921.411478:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617059200:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074921.411513:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617059200:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 34us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074921.411521:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2558 00000100:00100000:3.0:1769074921.411539:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059328 00000100:00100000:3.0:1769074921.411543:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059328 00000100:00100000:3.0:1769074921.411567:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059456 00000100:00100000:3.0:1769074921.411570:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059456 00000100:00100000:3.0:1769074921.411590:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1619 00000100:00100000:3.0:1769074921.411596:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617059328:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074921.411621:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617059328:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (553us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074921.411626:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1619 00000100:00100000:3.0:1769074921.411653:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1620 00000100:00100000:3.0:1769074921.411660:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617059456:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074921.411683:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617059456:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (552us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074921.411688:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1620 00000100:00080000:3.0:1769074921.411860:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074921.411865:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074921.411868:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074921.411869:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1769074921.885224:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074922.397755:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625276544:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074922.397902:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074922.398031:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276544 00000100:00100000:1.0:1769074922.398043:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276544 00000100:00100000:1.0:1769074922.398123:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2782 00000100:00100000:1.0:1769074922.398134:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625276544:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074922.398192:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:3.0:1769074922.402203:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625276672:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074922.402324:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074922.402743:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276672 00000100:00100000:0.0:1769074922.402756:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276672 00000100:00100000:0.0:1769074922.402858:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2783 00000100:00100000:0.0:1769074922.402870:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625276672:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074922.402931:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:0.0:1769074922.403003:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625276672:12345-0@lo:41:osp-pre-1-0.0 Request processed in 136us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074922.403011:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2783 00000100:00100000:0.0:1769074922.403081:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625276672:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074922.406708:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625276544:12345-0@lo:41:osp-pre-0-1.0 Request processed in 8559us (8790us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074922.406729:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2782 00000100:00100000:2.0:1769074922.407162:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54475ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625276544:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1769074922.909105:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074922.909863:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625276800:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074922.909965:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074922.910049:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074922.910057:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625276928:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074922.910082:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074922.910123:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276800 00000100:00100000:1.0:1769074922.910134:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276800 00000100:00100000:1.0:1769074922.910215:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625276928 00000100:00100000:1.0:1769074922.910220:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625276928 00000100:00100000:2.0:1769074922.910397:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625277056:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074922.910429:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074922.910586:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1284 00000100:00100000:1.0:1769074922.910600:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625276800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074922.910635:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074922.910684:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625276800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (720us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074922.910692:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1284 00000100:00100000:1.0:1769074922.910712:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277056 00000100:00100000:1.0:1769074922.910715:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277056 00000100:00100000:1.0:1769074922.910729:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1285 00000100:00100000:1.0:1769074922.910734:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625276928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074922.910749:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074922.910774:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625276928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 40us (692us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074922.910780:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1285 00000100:00100000:1.0:1769074922.910803:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1286 00000100:00100000:1.0:1769074922.910831:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625277056:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074922.910841:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074922.910867:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625277056:12345-0@lo:13:osp-pre-0-1.0 Request processed in 38us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074922.910876:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1286 00000100:00080000:1.0:1769074922.910899:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074922.910903:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074922.910965:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625276928:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074922.910980:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625276800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074922.911156:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625277056:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074923.423468:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625277184:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074923.423577:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074923.423754:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277184 00000100:00100000:1.0:1769074923.423774:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277184 00000100:00100000:1.0:1769074923.423898:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1287 00000100:00100000:1.0:1769074923.423910:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625277184:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1769074923.423941:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074923.424003:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625277184:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074923.424012:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1287 00000100:00100000:1.0:1769074923.424090:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625277184:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1769074923.424703:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074923.935085:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074924.958362:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1769074924.965112:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968380 is committed 00000001:00080000:1.0:1769074924.965159:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874646 is committed 00000001:00080000:0.0:1769074924.965426:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841313 is committed 00000100:00100000:3.0:1769074925.469558:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074925.472482:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625277440:0@lo:400:kworker.0 00000100:00100000:3.0:1769074925.472565:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074925.472682:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625277312:0@lo:400:kworker.0 00000100:00100000:3.0:1769074925.472713:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074925.472840:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277312 00000100:00100000:1.0:1769074925.472849:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277312 00000100:00100000:1.0:1769074925.472905:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:1.0:1769074925.472914:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b0ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625277312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.472971:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b0ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625277312:12345-0@lo:400:kworker.0 Request processed in 58us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.472978:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1769074925.476409:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f2b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625277312:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.478707:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625277568:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.478785:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074925.478837:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625278080:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.478858:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074925.478875:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625278592:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.478898:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074925.478925:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1769074925.478931:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625277696:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.478953:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074925.478971:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625278208:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.478990:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074925.479005:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625278720:0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.479037:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074925.479078:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277568 00000100:00100000:2.0:1769074925.479085:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277568 00000100:00100000:2.0:1769074925.479147:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278080 00000100:00100000:2.0:1769074925.479150:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278080 00000100:00100000:2.0:1769074925.479184:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278208 00000100:00100000:2.0:1769074925.479187:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278208 00000100:00100000:2.0:1769074925.479204:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2559 00000100:00100000:2.0:1769074925.479212:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625277568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.479263:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625277568:12345-0@lo:400:kworker.0 Request processed in 52us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074925.479270:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2559 00000100:00100000:2.0:1769074925.479289:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2560 00000100:00100000:2.0:1769074925.479295:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625278080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.479322:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625278080:12345-0@lo:400:kworker.0 Request processed in 28us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074925.479328:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2560 00000100:00100000:2.0:1769074925.479338:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2561 00000100:00100000:2.0:1769074925.479344:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625278208:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1769074925.479511:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074925.479515:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074925.479529:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625277952:0@lo:400:kworker.0 00000100:00100000:3.0:1769074925.479559:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074925.479577:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625278464:0@lo:400:kworker.0 00000100:00100000:3.0:1769074925.479602:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074925.479679:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277952 00000100:00100000:1.0:1769074925.479685:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277952 00000100:00100000:1.0:1769074925.479711:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2562 00000100:00100000:1.0:1769074925.479717:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625277952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.479764:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625277952:12345-0@lo:400:kworker.0 Request processed in 47us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.479770:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2562 00000100:00080000:1.0:1769074925.479787:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074925.479837:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278592 00000100:00100000:1.0:1769074925.479841:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278592 00000100:00100000:1.0:1769074925.479871:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277696 00000100:00100000:1.0:1769074925.479874:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277696 00000100:00100000:1.0:1769074925.479892:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278720 00000100:00100000:1.0:1769074925.479895:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278720 00000100:00100000:1.0:1769074925.479916:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1621 00000100:00100000:1.0:1769074925.479922:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625278592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.479954:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625278592:12345-0@lo:400:kworker.0 Request processed in 32us (1055us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.479960:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1621 00000100:00100000:1.0:1769074925.479972:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1622 00000100:00100000:1.0:1769074925.479977:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625277696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480003:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625277696:12345-0@lo:400:kworker.0 Request processed in 26us (1049us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.480008:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1622 00000100:00100000:1.0:1769074925.480019:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1623 00000100:00100000:1.0:1769074925.480024:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625278720:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480050:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625278720:12345-0@lo:400:kworker.0 Request processed in 26us (1012us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.480055:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1623 00000100:00080000:1.0:1769074925.480082:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074925.480087:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074925.480099:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656db800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625277824:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480125:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074925.480141:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625278336:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480160:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074925.480175:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625278848:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480193:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074925.480219:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277824 00000100:00100000:1.0:1769074925.480223:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277824 00000100:00100000:1.0:1769074925.480247:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1624 00000100:00100000:1.0:1769074925.480253:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625277824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480284:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625277824:12345-0@lo:400:kworker.0 Request processed in 32us (159us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.480290:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1624 00000100:00100000:1.0:1769074925.480306:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278336 00000100:00100000:1.0:1769074925.480309:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278336 00000100:00100000:1.0:1769074925.480325:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278848 00000100:00100000:1.0:1769074925.480327:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278848 00000100:00100000:1.0:1769074925.480339:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2563 00000100:00100000:1.0:1769074925.480345:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625278336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480388:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625278336:12345-0@lo:400:kworker.0 Request processed in 43us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.480394:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2563 00000100:00100000:1.0:1769074925.480406:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2564 00000100:00100000:1.0:1769074925.480411:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625278848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480436:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625278848:12345-0@lo:400:kworker.0 Request processed in 24us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.480441:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2564 00000100:00080000:1.0:1769074925.480456:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074925.480458:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074925.480482:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 17179874646 gen 2 00000004:00080000:1.0:1769074925.480487:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54920a680, transno 17179874645 00000004:00080000:1.0:1769074925.480497:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549208380, transno 17179874646 00000100:00100000:1.0:1769074925.480503:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656db800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625277824:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480514:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625278336:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480522:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625278848:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480695:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 21474841313 gen 2 00000004:00080000:1.0:1769074925.480700:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b549f29500, transno 21474841313 00000100:00100000:1.0:1769074925.480710:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625277696:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480720:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625278720:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480730:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625277568:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480739:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625278080:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.480762:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625278592:0@lo:400:kworker.0 00000100:00100000:1.0:1769074925.481932:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625278208:12345-0@lo:400:kworker.0 Request processed in 2581us (2933us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074925.481943:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2561 00000100:00100000:2.0:1769074925.482415:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625278208:0@lo:400:kworker.0 00000100:00100000:3.0:1769074925.482757:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625277952:0@lo:400:kworker.0 00000100:00100000:0.0:1769074925.482890:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625277440 00000100:00100000:0.0:1769074925.482898:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625277440 00000100:00100000:0.0:1769074925.482982:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278464 00000100:00100000:0.0:1769074925.482986:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278464 00000100:00100000:2.0:1769074925.486141:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2784 00000100:00100000:2.0:1769074925.486167:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625277440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.486278:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625277440:12345-0@lo:400:kworker.0 Request processed in 123us (13712us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074925.486285:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2784 00000100:00100000:2.0:1769074925.486309:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2785 00000100:00100000:2.0:1769074925.486317:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625278464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074925.486345:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625278464:12345-0@lo:400:kworker.0 Request processed in 30us (6742us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074925.486350:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2785 00000100:00100000:3.0:1769074925.486611:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625278464:0@lo:400:kworker.0 00000100:00100000:3.0:1769074925.486624:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625277440:0@lo:400:kworker.0 00010000:00010000:2.0:1769074925.982006:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074926.532642:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074926.532694:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074926.532740:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074926.532776:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074926.532863:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074926.532893:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059968 00000100:00100000:1.0:1769074926.532903:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059968 00000100:00100000:1.0:1769074926.532959:0:9048:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060096 00000100:00100000:1.0:1769074926.532963:0:9048:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060096 00000100:00100000:1.0:1769074926.532986:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1625 00000100:00100000:1.0:1769074926.532995:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617059968:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074926.533065:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617059968:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 71us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074926.533086:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1625 00000100:00100000:1.0:1769074926.533104:0:9048:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1626 00000100:00100000:1.0:1769074926.533110:0:9048:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617060096:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074926.533134:0:9048:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617060096:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 24us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074926.533140:0:9048:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1626 00000100:00100000:1.0:1769074926.533163:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059712 00000100:00100000:1.0:1769074926.533166:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059712 00000100:00100000:1.0:1769074926.533194:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059840 00000100:00100000:1.0:1769074926.533197:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059840 00000100:00100000:1.0:1769074926.533218:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2565 00000100:00100000:1.0:1769074926.533316:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007617059712:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074926.533348:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2408:x1855007617059712:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 124us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074926.533355:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2565 00000100:00100000:1.0:1769074926.533367:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2566 00000100:00100000:1.0:1769074926.533372:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617059840:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074926.533395:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617059840:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 23us (619us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074926.533401:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2566 00000100:00080000:1.0:1769074926.533421:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074926.533425:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074926.533428:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074926.533430:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074926.533712:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617059584 00000100:00100000:2.0:1769074926.533718:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617059584 00000100:00100000:2.0:1769074926.533771:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 403 00000100:00100000:2.0:1769074926.533779:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681b09050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617059584:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074926.533853:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681b09050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617059584:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 74us (1111us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074926.533861:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 403 00010000:00010000:2.0:1769074927.006829:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074927.517834:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074927.517859:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625278976:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074927.517944:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625279104:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074927.517949:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074927.517988:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074927.518014:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625278976 00000100:00100000:1.0:1769074927.518022:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625278976 00000100:00100000:1.0:1769074927.518065:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279104 00000100:00100000:1.0:1769074927.518067:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279104 00000100:00100000:1.0:1769074927.518090:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2786 00000100:00100000:1.0:1769074927.518098:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625278976:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074927.518116:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2787 00000004:00000024:1.0:1769074927.518134:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:2.0:1769074927.518143:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625279104:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074927.518166:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:1.0:1769074927.518181:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625278976:12345-0@lo:41:osp-pre-1-0.0 Request processed in 81us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074927.518188:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2786 00000100:00100000:2.0:1769074927.518210:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625279104:12345-0@lo:41:osp-pre-0-1.0 Request processed in 84us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074927.518219:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2787 00000100:00100000:1.0:1769074927.518239:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625278976:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074927.518260:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625279104:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074928.029326:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625279232:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1769074928.029398:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074928.029740:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279232 00000100:00100000:1.0:1769074928.029749:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279232 00000100:00100000:1.0:1769074928.029833:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1288 00000100:00100000:1.0:1769074928.029844:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625279232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074928.029874:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074928.029928:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625279232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074928.029936:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1288 00000100:00100000:1.0:1769074928.029980:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625279232:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1769074928.030240:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074928.030403:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625279360:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1769074928.030426:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074928.030460:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074928.030928:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074928.030939:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625279488:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074928.030957:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279360 00000100:00100000:2.0:1769074928.030964:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279360 00000100:00100000:3.0:1769074928.030970:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074928.031011:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279488 00000100:00100000:2.0:1769074928.031015:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279488 00000100:00080000:3.0:1769074928.031023:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074928.031028:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074928.031040:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1289 00000100:00100000:2.0:1769074928.031053:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625279360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074928.031077:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074928.031117:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625279360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (658us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074928.031123:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1289 00000100:00100000:2.0:1769074928.031136:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1290 00000100:00100000:2.0:1769074928.031141:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625279488:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074928.031150:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074928.031169:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625279488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 29us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074928.031175:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1290 00000100:00100000:3.0:1769074928.033789:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625279488:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074928.033846:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625279360:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074928.541237:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625279616:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074928.541311:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074928.541653:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279616 00000100:00100000:2.0:1769074928.541662:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279616 00000100:00100000:2.0:1769074928.541739:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1291 00000100:00100000:2.0:1769074928.541748:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625279616:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074928.541771:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 51836224 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074928.541847:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625279616:12345-0@lo:13:osp-pre-1-1.0 Request processed in 100us (535us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074928.541857:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1291 00000100:00080000:3.0:1769074928.541929:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074928.542156:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625279616:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1769074929.054733:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074930.077090:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074930.589998:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625280000:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590077:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074930.590140:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625280512:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590174:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074930.590192:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625281024:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590249:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074930.590281:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1769074930.590287:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625279872:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590308:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074930.590330:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625280384:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590348:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074930.590363:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625280896:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590379:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074930.590405:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280000 00000100:00100000:1.0:1769074930.590413:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280000 00000100:00100000:1.0:1769074930.590454:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280512 00000100:00100000:1.0:1769074930.590456:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280512 00000100:00100000:1.0:1769074930.590477:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280384 00000100:00100000:1.0:1769074930.590480:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280384 00000100:00100000:1.0:1769074930.590495:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2567 00000100:00100000:1.0:1769074930.590501:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625280000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590549:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1855007625280000:12345-0@lo:400:kworker.0 Request processed in 47us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074930.590559:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2567 00000100:00100000:1.0:1769074930.590577:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2568 00000100:00100000:1.0:1769074930.590583:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625280512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590610:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1855007625280512:12345-0@lo:400:kworker.0 Request processed in 27us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074930.590616:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2568 00000100:00100000:1.0:1769074930.590629:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2569 00000100:00100000:1.0:1769074930.590637:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625280384:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1769074930.590637:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074930.590644:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074930.590680:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1855007625280384:12345-0@lo:400:kworker.0 Request processed in 44us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074930.590686:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2569 00000100:00100000:3.0:1769074930.590701:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625281024 00000100:00100000:3.0:1769074930.590708:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625281024 00000100:00100000:1.0:1769074930.590739:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3650:1855007625280384:0@lo:400:kworker.0 00000100:00100000:3.0:1769074930.590751:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1627 00000100:00100000:1.0:1769074930.590755:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3650:1855007625280000:0@lo:400:kworker.0 00000100:00100000:3.0:1769074930.590763:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625281024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590768:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3650:1855007625280512:0@lo:400:kworker.0 00000100:00100000:3.0:1769074930.590840:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625281024:12345-0@lo:400:kworker.0 Request processed in 80us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074930.590844:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625281024:0@lo:400:kworker.0 00000100:00100000:3.0:1769074930.590852:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1627 00000100:00080000:3.0:1769074930.590886:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074930.590888:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279872 00000100:00100000:1.0:1769074930.590894:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279872 00000100:00100000:1.0:1769074930.590933:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280896 00000100:00100000:1.0:1769074930.590937:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280896 00000100:00100000:1.0:1769074930.590955:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2788 00000100:00100000:1.0:1769074930.590961:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625279872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.590998:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625279872:12345-0@lo:400:kworker.0 Request processed in 37us (688us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074930.591004:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2788 00000100:00100000:1.0:1769074930.591017:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2789 00000100:00100000:1.0:1769074930.591024:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625280896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.591065:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625280896:12345-0@lo:400:kworker.0 Request processed in 41us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074930.591071:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2789 00000100:00100000:1.0:1769074930.591096:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625279872:0@lo:400:kworker.0 00000100:00100000:1.0:1769074930.591105:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54510b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625280896:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591580:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625280128:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591618:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591654:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625280640:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591685:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591707:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625281152:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591727:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591748:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1769074930.591754:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625279744:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591780:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591801:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625280256:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591845:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591864:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625280768:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591896:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591913:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625281280:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.591930:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074930.591959:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625279744 00000100:00100000:2.0:1769074930.591965:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625279744 00000100:00100000:2.0:1769074930.591994:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1769074930.592001:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fb9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625279744:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.592039:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fb9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3651:x1855007625279744:12345-0@lo:400:kworker.0 Request processed in 38us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074930.592048:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1769074930.592077:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280128 00000100:00100000:2.0:1769074930.592080:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280128 00000100:00100000:2.0:1769074930.592101:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625281152 00000100:00100000:2.0:1769074930.592103:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625281152 00000100:00100000:2.0:1769074930.592116:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280256 00000100:00100000:2.0:1769074930.592118:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280256 00000100:00100000:2.0:1769074930.592134:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1628 00000100:00100000:2.0:1769074930.592140:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625280128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592175:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280640 00000100:00100000:2.0:1769074930.592182:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625280128:12345-0@lo:400:kworker.0 Request processed in 40us (562us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074930.592183:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280640 00000100:00100000:2.0:1769074930.592192:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1628 00000100:00100000:2.0:1769074930.592206:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1629 00000100:00100000:2.0:1769074930.592212:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625281152:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592223:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625280768 00000100:00100000:0.0:1769074930.592228:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625280768 00000100:00100000:2.0:1769074930.592237:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625281152:12345-0@lo:400:kworker.0 Request processed in 25us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074930.592244:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1629 00000100:00100000:0.0:1769074930.592254:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625281280 00000100:00100000:0.0:1769074930.592258:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625281280 00000100:00100000:2.0:1769074930.592259:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1630 00000100:00100000:2.0:1769074930.592269:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625280256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592284:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2570 00000100:00100000:2.0:1769074930.592293:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3651:x1855007625280256:12345-0@lo:400:kworker.0 Request processed in 25us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074930.592296:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625280640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.592304:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1630 00000100:00100000:2.0:1769074930.592335:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:025c5dec-f591-4c93-9b92-fac36a3bdc36:3651:1855007625279744:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.592349:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625280256:0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592354:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625280640:12345-0@lo:400:kworker.0 Request processed in 59us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074930.592363:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625280128:0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592364:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2570 00000100:00100000:2.0:1769074930.592379:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625280640:0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592391:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2571 00000100:00100000:2.0:1769074930.592392:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625281152:0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592404:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625280768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.592432:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2572 00000100:00100000:2.0:1769074930.592440:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625281280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074930.592448:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1855007625280768:12345-0@lo:400:kworker.0 Request processed in 46us (550us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074930.592455:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2571 00000100:00100000:2.0:1769074930.592474:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1855007625281280:12345-0@lo:400:kworker.0 Request processed in 34us (541us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074930.592481:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2572 00000100:00100000:2.0:1769074930.592508:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b545108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3651:1855007625280768:0@lo:400:kworker.0 00000100:00100000:2.0:1769074930.592515:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3651:1855007625281280:0@lo:400:kworker.0 00000100:00080000:3.0:1769074930.593178:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074930.593183:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074930.593185:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074930.593188:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1769074931.108184:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074931.651523:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074931.651574:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074931.651609:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074931.651627:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074931.651660:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074931.651768:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060608 00000100:00100000:0.0:1769074931.651780:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060608 00000100:00100000:0.0:1769074931.651868:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060736 00000100:00100000:0.0:1769074931.651872:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060736 00000100:00100000:0.0:1769074931.651908:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1631 00000100:00100000:0.0:1769074931.651917:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617060608:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074931.651994:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617060608:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 77us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074931.652003:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1631 00000100:00100000:0.0:1769074931.652020:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1632 00000100:00100000:0.0:1769074931.652026:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617060736:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074931.652068:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617060736:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 41us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074931.652074:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1632 00000100:00100000:1.0:1769074931.652587:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060352 00000100:00100000:1.0:1769074931.652593:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060352 00000100:00100000:1.0:1769074931.652638:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060480 00000100:00100000:1.0:1769074931.652651:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060480 00000100:00100000:1.0:1769074931.652675:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2573 00000100:00100000:1.0:1769074931.652682:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007617060352:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074931.652727:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:2407:x1855007617060352:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 45us (1214us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074931.652733:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2573 00000100:00100000:1.0:1769074931.652761:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2574 00000100:00100000:1.0:1769074931.652768:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617060480:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074931.652794:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2407:x1855007617060480:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 27us (1166us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074931.652800:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2574 00000100:00100000:2.0:1769074931.653009:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060224 00000100:00100000:2.0:1769074931.653014:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060224 00000100:00100000:2.0:1769074931.653060:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 405 00000100:00100000:2.0:1769074931.653068:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fba850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617060224:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074931.653104:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fba850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617060224:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 37us (1494us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074931.653110:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 405 00000100:00080000:3.0:1769074931.653687:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1769074931.653692:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769074931.653695:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074931.653698:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1769074932.125151:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074932.637990:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625281408:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074932.638073:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074932.638304:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625281408 00000100:00100000:2.0:1769074932.638313:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625281408 00000100:00100000:2.0:1769074932.638369:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2790 00000100:00100000:2.0:1769074932.638378:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625281408:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1769074932.638412:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:2.0:1769074932.638471:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625281408:12345-0@lo:41:osp-pre-1-0.0 Request processed in 94us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074932.638481:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2790 00000100:00100000:3.0:1769074932.638581:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625281408:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074932.638742:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625281536:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074932.638774:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074932.638876:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625281536 00000100:00100000:0.0:1769074932.638892:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625281536 00000100:00100000:0.0:1769074932.638925:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2791 00000100:00100000:0.0:1769074932.638933:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625281536:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074932.638951:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:0.0:1769074932.638993:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625281536:12345-0@lo:41:osp-pre-0-1.0 Request processed in 61us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074932.639000:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2791 00000100:00100000:1.0:1769074932.639585:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6656d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625281536:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074932.760762:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074932.761041:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617060992 00000100:00100000:3.0:1769074932.761052:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617060992 00000100:00100000:3.0:1769074932.761114:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13652 00000100:00100000:3.0:1769074932.761124:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68488:x1855007617060992:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074932.761143:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074932.761168:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e26 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9d6 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074932.761246:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.761259:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761272:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761283:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.761299:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761306:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761382:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761389:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761395:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761401:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761407:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.761411:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761422:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761434:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:3.0:1769074932.761437:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.761441:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761447:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e2d lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761460:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761465:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761471:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761475:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.761477:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761482:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761489:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea100) 00010000:00010000:3.0:1769074932.761491:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.761494:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761500:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e34 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.761532:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e26 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9d6 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074932.761542:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:3.0:1769074932.761544:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074932.761548:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e26 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9d6 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074932.761558:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febbc00, rc 0) 00000100:00100000:3.0:1769074932.761865:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68488:x1855007617060992:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 686us (1059us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074932.761872:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13652 00000100:00100000:2.0:1769074932.768400:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074932.768860:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617061248 00000100:00100000:1.0:1769074932.768871:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617061248 00000100:00100000:1.0:1769074932.768940:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13653 00000100:00100000:1.0:1769074932.768949:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+23:68488:x1855007617061248:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074932.768967:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074932.768988:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55e3b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9dd expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1769074932.769149:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20c:0x0]: returning size 4096 00010000:00010000:1.0:1769074932.769260:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55e3b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9dd expref: 25 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074932.769273:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:1.0:1769074932.769276:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1769074932.769281:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549243e00/0x7fe2323adfd55e3b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9dd expref: 25 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074932.769294:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243e00, rc 0) 00000100:00100000:1.0:1769074932.769362:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:68488:x1855007617061248:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 414us (973us total) trans 8589944142 rc 301/301 00000100:00100000:1.0:1769074932.769370:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13653 00000100:00100000:1.0:1769074932.782550:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074932.782783:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617061376 00000100:00100000:3.0:1769074932.782794:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617061376 00000100:00100000:3.0:1769074932.782887:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13654 00000100:00100000:3.0:1769074932.782897:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68488:x1855007617061376:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074932.782985:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.783003:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783015:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783027:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.783033:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783040:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783107:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.783116:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783123:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:3.0:1769074932.783164:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00000004:00200000:3.0:1769074932.783169:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074932.783181:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783196:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783201:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783206:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783212:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783217:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.783220:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783227:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783240:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea800) 00010000:00010000:3.0:1769074932.783243:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.783247:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783252:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e49 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783270:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783275:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783279:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.783283:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.783286:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789763:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789785:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea100) 00010000:00010000:3.0:1769074932.789789:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.789795:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789801:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e50 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789838:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789846:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789853:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789859:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.789862:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789867:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789873:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074932.789876:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.789879:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.789884:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e57 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:3.0:1769074932.789999:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68488:x1855007617061376:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 7098us (7453us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074932.790010:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13654 00000100:00100000:0.0:1769074932.803697:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.803836:0:6516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617061632 00000100:00100000:2.0:1769074932.803849:0:6516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617061632 00000100:00100000:2.0:1769074932.803915:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4752 00000100:00100000:2.0:1769074932.803926:0:6516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:68488:x1855007617061632:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074932.804170:0:6516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:68488:x1855007617061632:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 246us (481us total) trans 8589944143 rc 0/0 00000100:00100000:2.0:1769074932.804178:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4752 00000100:00100000:2.0:1769074932.856378:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074932.856503:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617061888 00000100:00100000:3.0:1769074932.856513:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617061888 00000100:00100000:3.0:1769074932.856575:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13655 00000100:00100000:3.0:1769074932.856594:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68493:x1855007617061888:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074932.856629:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074932.856652:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e5e lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9e4 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074932.856730:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.856744:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856754:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856766:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074932.856772:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856778:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856895:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856902:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856910:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856917:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856922:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.856926:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856938:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856950:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea100) 00010000:00010000:3.0:1769074932.856953:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.856958:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856962:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febac00/0x7fe2323adfd55e65 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856985:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856991:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856995:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.856999:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074932.857001:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.857006:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.857013:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea800) 00010000:00010000:3.0:1769074932.857016:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074932.857018:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.857023:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55e6c lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074932.857055:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e5e lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9e4 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074932.857064:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea100) 00010000:00010000:3.0:1769074932.857066:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074932.857070:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb400/0x7fe2323adfd55e5e lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9e4 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074932.857080:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:2.0:1769074932.857284:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68493:x1855007617061888:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 689us (914us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074932.857292:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13655 00000100:00100000:1.0:1769074932.889001:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.889296:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617062144 00000100:00100000:2.0:1769074932.889307:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617062144 00000100:00100000:2.0:1769074932.889372:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13656 00000100:00100000:2.0:1769074932.889382:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:68493:x1855007617062144:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074932.889407:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074932.889429:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55e73 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9eb expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.889585:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55e7a lrc: 3/1,0 mode: --/CR res: [0x200000405:0x20c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.889600:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55e7a lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x20c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.889612:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55e7a lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x20c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT0 00000004:00200000:2.0:1769074932.889632:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20c:0x0]: returning size 4096 00010000:00010000:2.0:1769074932.889734:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd55e73 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9eb expref: 28 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.889755:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88600/0x7fe2323adfd55e7a lrc: 3/0,0 mode: CR/CR res: [0x200000405:0x20c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9eb expref: 27 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.889764:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20c:0x0].0x0 (ffff89b64a19d300) 00010000:00010000:2.0:1769074932.889767:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1769074932.889770:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88600, rc 0) 00000100:00100000:2.0:1769074932.890002:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:68493:x1855007617062144:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 620us (1010us total) trans 8589944144 rc 0/0 00000100:00100000:2.0:1769074932.890010:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13656 00000100:00100000:0.0:1769074932.901455:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.901528:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617062272 00000100:00100000:2.0:1769074932.901538:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617062272 00000100:00100000:2.0:1769074932.901600:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13657 00000100:00100000:2.0:1769074932.901609:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68493:x1855007617062272:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074932.901696:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.901712:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901723:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901750:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.901762:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901769:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901902:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.901911:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901918:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:2.0:1769074932.901959:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00000004:00200000:2.0:1769074932.901964:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074932.901975:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901988:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.901994:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902000:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902007:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902013:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.902017:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902025:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902037:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effae00) 00010000:00010000:2.0:1769074932.902040:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.902044:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902049:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55e88 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902067:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902073:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902078:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902083:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.902085:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902091:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902098:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa400) 00010000:00010000:2.0:1769074932.902101:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.902103:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902109:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55e8f lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902120:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902125:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902130:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902135:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.902137:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902143:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902150:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074932.902152:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.902154:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.902158:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e96 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074932.902222:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68493:x1855007617062272:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 613us (776us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074932.902230:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13657 00000100:00100000:0.0:1769074932.947769:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.948008:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617062528 00000100:00100000:2.0:1769074932.948019:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617062528 00000100:00100000:2.0:1769074932.948086:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13658 00000100:00100000:2.0:1769074932.948097:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68496:x1855007617062528:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074932.948129:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074932.948152:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e9d lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9f2 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.948224:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074932.948237:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948249:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948262:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074932.948270:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948275:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948342:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948350:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948357:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948363:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948369:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.948374:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948381:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948393:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074932.948397:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.948401:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948406:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ea4 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948419:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948425:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948430:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948435:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.948437:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948443:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948449:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effae00) 00010000:00010000:2.0:1769074932.948452:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.948454:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948462:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eab lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.948495:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e9d lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9f2 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.948505:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074932.948507:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074932.948511:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55e9d lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9f2 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.948522:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074932.948583:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b544759500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68496:x1855007617062528:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 488us (826us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074932.948590:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13658 00000100:00100000:1.0:1769074932.957236:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.957410:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617062656 00000100:00100000:2.0:1769074932.957420:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617062656 00000100:00100000:2.0:1769074932.957483:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13659 00000100:00100000:2.0:1769074932.957494:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68496:x1855007617062656:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074932.957569:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.957586:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957598:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957610:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.957617:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957634:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957705:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:2.0:1769074932.957715:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957722:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:2.0:1769074932.957771:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00000004:00200000:2.0:1769074932.957776:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074932.957787:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957801:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957843:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957848:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957854:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957859:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.957863:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957869:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957880:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa400) 00010000:00010000:2.0:1769074932.957883:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.957886:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957891:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55eb2 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957908:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957914:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957920:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957925:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.957929:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957933:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957939:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effae00) 00010000:00010000:2.0:1769074932.957942:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.957944:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957949:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eb9 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957960:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957965:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.957970:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.958202:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.958206:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.958212:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.958220:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074932.958222:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.958225:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:2.0:1769074932.958232:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec0 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074932.958308:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54510b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68496:x1855007617062656:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 815us (1079us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074932.958316:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13659 00000100:00100000:0.0:1769074932.992145:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.992246:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617062912 00000100:00100000:2.0:1769074932.992257:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617062912 00000100:00100000:2.0:1769074932.992320:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13660 00000100:00100000:2.0:1769074932.992329:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68498:x1855007617062912:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074932.992352:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074932.992373:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec7 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879c9f9 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.992454:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074932.992469:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992480:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992493:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074932.992498:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992504:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992575:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992581:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992587:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992593:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992598:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.992603:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992609:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992633:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effae00) 00010000:00010000:2.0:1769074932.992637:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.992641:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992648:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55ece lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992662:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992668:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992674:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992678:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074932.992681:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992687:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992693:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa400) 00010000:00010000:2.0:1769074932.992696:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074932.992698:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992707:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ed5 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074932.992749:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec7 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9f9 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.992759:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effae00) 00010000:00010000:2.0:1769074932.992761:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074932.992765:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ec7 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9f9 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074932.992775:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89200, rc 0) 00000100:00100000:2.0:1769074932.992894:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68498:x1855007617062912:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 565us (758us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074932.992902:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13660 00000100:00100000:1.0:1769074932.997550:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074932.997659:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617063040 00000100:00100000:2.0:1769074932.997669:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617063040 00000100:00100000:0.0:1769074932.997851:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13661 00000100:00100000:0.0:1769074932.997863:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68498:x1855007617063040:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074932.997942:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074932.997958:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998048:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998061:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074932.998068:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998074:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998507:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074932.998517:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998523:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:0.0:1769074932.998560:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00000004:00200000:0.0:1769074932.998564:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074932.998575:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998588:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998598:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998603:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998609:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998615:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074932.998629:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998636:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998648:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b67a317a00) 00010000:00010000:0.0:1769074932.998651:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074932.998655:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998661:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd55edc lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998679:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998685:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998691:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998696:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074932.998699:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998704:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998710:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b67a317700) 00010000:00010000:0.0:1769074932.998712:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074932.998715:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998720:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd55ee3 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998741:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998747:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998752:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998757:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549135800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074932.998760:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998765:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998772:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074932.998774:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074932.998777:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074932.998785:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135800/0x7fe2323adfd55eea lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:0.0:1769074932.999396:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545109180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68498:x1855007617063040:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1530us (1849us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074932.999406:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13661 00000100:00100000:0.0:1769074933.041424:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.041536:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617063168 00000100:00100000:2.0:1769074933.041547:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617063168 00000100:00100000:2.0:1769074933.041633:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13662 00000100:00100000:2.0:1769074933.041643:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68501:x1855007617063168:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.041663:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.041686:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ef1 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca00 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.041753:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.041766:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.041776:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.041790:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.041797:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.041804:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.041925:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.041935:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.041941:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:2.0:1769074933.041993:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00000004:00200000:2.0:1769074933.041998:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074933.042016:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042027:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042046:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042053:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042062:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042068:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042074:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.042079:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042086:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042098:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effae00) 00010000:00010000:2.0:1769074933.042101:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.042106:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042112:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55ef8 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042125:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042131:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042136:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042141:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.042144:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042152:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042159:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa400) 00010000:00010000:2.0:1769074933.042161:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.042163:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042169:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55eff lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.042198:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89200/0x7fe2323adfd55ef1 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca00 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.042212:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca00 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.042218:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.042220:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769074933.042222:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074933.042276:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545388e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617063168:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 634us (862us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.042283:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13662 00000100:00080000:3.0:1769074933.043366:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074933.051354:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.052320:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617063296 00000100:00100000:3.0:1769074933.052332:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617063296 00000100:00100000:3.0:1769074933.052415:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14546 00000100:00100000:3.0:1769074933.052425:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2408:x1855007617063296:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074933.052442:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074933.052462:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca07 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1769074933.052486:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x36:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1769074933.052503:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.052505:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.052515:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 9 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074933.052533:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 9 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074933.052551:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b67febb400, rc 0) 00000100:00100000:3.0:1769074933.052628:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617063296:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 197us (1276us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.052635:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14546 00000100:00080000:0.0:1769074933.053634:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769074933.057876:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.058166:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617063552 00000100:00100000:1.0:1769074933.058177:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617063552 00000100:00100000:1.0:1769074933.058241:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13663 00000100:00100000:1.0:1769074933.058252:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617063552:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.058274:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.058296:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd55f14 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca0e expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.058420:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f1b lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074933.058434:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f1b lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.058444:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f1b lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:1.0:1769074933.058457:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00010000:00010000:1.0:1769074933.058579:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd55f14 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca0e expref: 13 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.058598:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f1b lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca0e expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.058606:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.058609:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074933.058613:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243800, rc 0) 00000100:00100000:1.0:1769074933.058689:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545389f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68501:x1855007617063552:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 439us (822us total) trans 4294968381 rc 0/0 00000100:00100000:1.0:1769074933.058697:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13663 00000100:00100000:2.0:1769074933.066263:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.066408:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617063680 00000100:00100000:0.0:1769074933.066417:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617063680 00000100:00100000:0.0:1769074933.066475:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1047 00000100:00100000:0.0:1769074933.066481:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617063680:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074933.066493:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074933.066514:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f1b lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca0e expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.066529:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074933.066532:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074933.066580:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617063680:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 100us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074933.066586:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1047 00000100:00100000:0.0:1769074933.075925:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.076605:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617063936 00000100:00100000:2.0:1769074933.076616:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617063936 00000100:00100000:2.0:1769074933.076684:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13664 00000100:00100000:2.0:1769074933.076693:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68501:x1855007617063936:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.076717:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.076744:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55f29 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca15 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.076882:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55f30 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.076896:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55f30 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.076907:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55f30 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:2.0:1769074933.076918:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00010000:00010000:2.0:1769074933.077026:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55f29 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca15 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.077045:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55f30 lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca15 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.077053:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.077056:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074933.077059:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89800, rc 0) 00000100:00100000:2.0:1769074933.078414:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:68501:x1855007617063936:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 416us (1192us total) trans 4294968382 rc 0/0 00000100:00100000:2.0:1769074933.078425:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13664 00000100:00100000:1.0:1769074933.088630:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.088675:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.088752:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064192 00000100:00100000:3.0:1769074933.088762:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064192 00000100:00100000:3.0:1769074933.088844:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064064 00000100:00100000:3.0:1769074933.088848:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064064 00000100:00100000:3.0:1769074933.088876:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14547 00000100:00100000:3.0:1769074933.088896:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617064192:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074933.088941:0:8396:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.088960:0:8396:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 8 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.088969:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074933.088981:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074933.088985:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.089021:0:8396:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 8 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.089035:0:8396:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 8 pid: 28519 timeout: 2020 lvb_type: 0 00000100:00100000:3.0:1769074933.089054:0:8396:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5461e4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625281664:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074933.089382:0:8396:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b492480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074933.089404:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14548 00000100:00100000:3.0:1769074933.089413:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617064064:12345-192.168.204.60@tcp:1:lfs.0 00010000:00010000:3.0:1769074933.089437:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.089449:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 9 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074933.089455:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.089456:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.089458:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.089480:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 9 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1769074933.089488:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 9 pid: 8396 timeout: 2020 lvb_type: 1 00000100:00100000:3.0:1769074933.089498:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625281792:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074933.089523:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b492540 going to sleep for 11 seconds 00000100:00080000:0.0:1769074933.089694:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074933.092739:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:36516:1855007625281792:192.168.204.60@tcp:104: 00010000:00010000:1.0:1769074933.092769:0:36516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074933.092921:0:8396:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5461e4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8396:1855007625281664:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074933.092937:0:8396:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074933.099387:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.099541:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064320 00000100:00100000:2.0:1769074933.099551:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064320 00002000:00100000:2.0:1769074933.099599:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x38:0x0] (69206016->70254591) req@ffff89b54c844e00 x1855007617064320/t0(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:469/0 lens 488/0 e 0 to 0 dl 1769074944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074933.099614:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c844e00 with x1855007617064320 ext(69206016->70254591) 00010000:00010000:2.0:1769074933.099634:0:8404:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 9 pid: 28519 timeout: 2020 lvb_type: 0 00010000:00010000:2.0:1769074933.099648:0:8404:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 9 pid: 28519 timeout: 2020 lvb_type: 0 00002000:00010000:2.0:1769074933.099655:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54c844e00 00000100:00100000:2.0:1769074933.099678:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 817 00000100:00100000:2.0:1769074933.099687:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617064320:12345-192.168.204.60@tcp:4:dd.0 00000100:00100000:1.0:1769074933.101687:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.105703:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064448 00000100:00100000:2.0:1769074933.105719:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064448 00010000:00010000:2.0:1769074933.105781:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 10 pid: 8396 timeout: 2020 lvb_type: 1 00000100:00100000:2.0:1769074933.105863:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7234 00000100:00100000:2.0:1769074933.105872:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617064448:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074933.105898:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074933.105919:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 10 pid: 8396 timeout: 2020 lvb_type: 1 00010000:00010000:2.0:1769074933.105933:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 10 pid: 8396 timeout: 2020 lvb_type: 1 00010000:00010000:2.0:1769074933.105955:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febb400/0x7fe2323adfd55f0d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca07 expref: 10 pid: 8396 timeout: 2020 lvb_type: 1 00010000:00010000:2.0:1769074933.105971:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.105973:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.105978:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.105987:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.106013:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074933.106068:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617064448:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 197us (4389us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.106076:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7234 00010000:00010000:1.0:1769074933.106979:0:36516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.106990:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107012:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107020:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107028:0:36516:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107035:0:36516:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febac00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074933.107039:0:36516:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107047:0:36516:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107060:0:36516:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.107067:0:36516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b67febac00/0x7fe2323adfd55f45 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074933.107144:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617064064:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 17731us (18466us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.107153:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14548 00000100:00100000:2.0:1769074933.149347:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074933.149372:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d102300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625281920:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074933.151260:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074933.151339:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625281920 00000100:00100000:2.0:1769074933.151348:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625281920 00000100:00100000:2.0:1769074933.151403:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1292 00000100:00100000:2.0:1769074933.151412:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625281920:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074933.151846:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625282048:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074933.151900:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074933.151935:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625282048 00000100:00100000:1.0:1769074933.151940:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625282048 00000100:00100000:1.0:1769074933.151979:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1293 00000100:00100000:1.0:1769074933.151986:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625282048:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074933.152007:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074933.152052:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625282048:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.152059:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1293 00002000:00000024:2.0:1769074933.152612:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 60273984 pending 1073152 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074933.152665:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625281920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1254us (1413us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.152672:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1292 00000100:00080000:2.0:1769074933.152693:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769074933.152697:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1769074933.152752:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074933.152894:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625282176:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074933.152925:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074933.152983:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625282176 00000100:00100000:2.0:1769074933.152988:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625282176 00000100:00100000:2.0:1769074933.153023:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1294 00000100:00100000:2.0:1769074933.153030:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625282176:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074933.153042:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484922688 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074933.153073:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625282176:12345-0@lo:13:osp-pre-0-1.0 Request processed in 43us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.153080:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1294 00000100:00080000:2.0:1769074933.153092:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074933.156461:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625282048:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074933.156507:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d102300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625281920:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074933.157208:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625282176:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1769074933.172033:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617064320:12345-192.168.204.60@tcp:4:dd.0 Request processed in 72337us (72649us total) trans 17179874647 rc 0/0 00000100:00100000:2.0:1769074933.172047:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 817 00002000:00100000:2.0:1769074933.172061:0:8404:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x38:0x0] (69206016->70254591) req@ffff89b54c844e00 x1855007617064320/t17179874647(0) o4->249ac4ab-c06c-4166-99d5-c4c7e1cc2791@192.168.204.60@tcp:469/0 lens 488/448 e 0 to 0 dl 1769074944 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1769074933.172072:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b54c844e00 with x1855007617064320 ext(69206016->70254591) 00010000:00010000:2.0:1769074933.172083:0:8404:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 11 pid: 28519 timeout: 2020 lvb_type: 0 00010000:00010000:2.0:1769074933.172187:0:8404:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 11 pid: 28519 timeout: 2021 lvb_type: 0 00002000:00010000:2.0:1769074933.172198:0:8404:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff89b54c844e00 00000100:00100000:0.0:1769074933.186523:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.186592:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064576 00000100:00100000:3.0:1769074933.186602:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064576 00010000:00010000:3.0:1769074933.186646:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 11 pid: 28519 timeout: 2021 lvb_type: 0 00000100:00100000:3.0:1769074933.186685:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7235 00000100:00100000:3.0:1769074933.186695:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2407:x1855007617064576:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074933.186711:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1769074933.186730:0:8424:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb size from disk: 0 -> 70254592 00002000:00010000:3.0:1769074933.186733:0:8424:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb mtime from disk: 0 -> 1769074919 00002000:00010000:3.0:1769074933.186736:0:8424:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb ctime from disk: 0 -> 1769074919 00002000:00010000:3.0:1769074933.186738:0:8424:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:3.0:1769074933.186744:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 11 pid: 28519 timeout: 2021 lvb_type: 0 00010000:00010000:3.0:1769074933.186758:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 11 pid: 28519 timeout: 2021 lvb_type: 0 00010000:00010000:3.0:1769074933.186782:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549242000/0x7fe2323adfd55e11 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 69206016->70254591) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c8 expref: 11 pid: 28519 timeout: 2021 lvb_type: 0 00010000:00010000:3.0:1769074933.186795:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.186797:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.186802:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.186846:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.186874:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074933.186928:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2407:x1855007617064576:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 234us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.186936:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7235 00010000:00010000:0.0:1769074933.187083:0:8396:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187097:0:8396:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187116:0:8396:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187124:0:8396:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187132:0:8396:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187139:0:8396:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbc00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074933.187143:0:8396:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187151:0:8396:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187162:0:8396:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.187168:0:8396:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febbc00/0x7fe2323adfd55f3e lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074933.187225:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617064192:12345-192.168.204.60@tcp:1:lfs.0 Request processed in 98331us (98615us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074933.187233:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14547 00000100:00100000:0.0:1769074933.195147:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.195438:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064832 00000100:00100000:3.0:1769074933.195449:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064832 00000100:00100000:3.0:1769074933.195510:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13665 00000100:00100000:3.0:1769074933.195521:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:68501:x1855007617064832:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:3.0:1769074933.195575:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1769074933.195605:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88e00/0x7fe2323adfd55f06 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca00 expref: 13 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.195637:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55e0a lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0xa/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9c1 expref: 12 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.195648:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074933.195652:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074933.195682:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074933.195694:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074933.195704:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00080000:3.0:1769074933.196150:0:6512:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x37:0x0], index 0: 0 00000040:00080000:3.0:1769074933.196209:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].68, 88 off13624 00000004:00080000:3.0:1769074933.196215:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1921038408174ns, next at 1833854042001ns, rc = 0 00000040:00080000:3.0:1769074933.196243:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].73, 88 off13968 00000004:00080000:3.0:1769074933.196247:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1921038440549ns, next at 1835465451293ns, rc = 0 00000040:00080000:3.0:1769074933.196255:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].74, 88 off14056 00000004:00080000:3.0:1769074933.196258:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1921038450862ns, next at 1835465451293ns, rc = 0 00000040:00080000:3.0:1769074933.196262:0:6512:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].69, 88 off13712 00000004:00080000:3.0:1769074933.196264:0:6512:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1921038457704ns, next at 1833854042001ns, rc = 0 00000001:00080000:3.0:1769074933.202111:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968383 is committed 00000004:00080000:3.0:1769074933.202127:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 68 committed 00000004:00080000:3.0:1769074933.202146:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 73 committed 00000004:00080000:3.0:1769074933.202160:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 72 committed 00000004:00080000:3.0:1769074933.202162:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 67 committed 00000004:00010000:3.0:1769074933.202224:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000002bc75657 reply state 000000009acab151 transno 4294968383 ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074933.202247:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074933.202257:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074933.202261:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:3.0:1769074933.202265:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074933.202269:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625282432:0@lo:2:osp-syn-1-1.0 00000100:00100000:1.0:1769074933.202338:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074933.202431:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625282304:0@lo:2:osp-syn-1-1.0 00010000:00080000:3.0:1769074933.202502:0:6512:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1769074933.202522:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617064832:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 7001us (7381us total) trans 4294968383 rc 0/0 00000100:00100000:3.0:1769074933.202532:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13665 00000100:00100000:0.0:1769074933.209429:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074933.209564:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1769074933.209579:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625282688:0@lo:2:osp-syn-0-1.0 00000100:00100000:0.0:1769074933.209641:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1769074933.209690:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.209701:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.209709:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.209715:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febb200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.209719:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.209728:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.209742:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074933.209746:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769074933.209751:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.209757:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b67febb200/0x7fe2323adfd55f4c lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:0.0:1769074933.210005:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625282432 00000100:00100000:0.0:1769074933.210015:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625282432 00000100:00100000:0.0:1769074933.210071:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625282304 00000100:00100000:0.0:1769074933.210074:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625282304 00000100:00100000:0.0:1769074933.210095:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625282688 00000100:00100000:0.0:1769074933.210098:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625282688 00000100:00100000:0.0:1769074933.210124:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14549 00000100:00100000:0.0:1769074933.210133:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3651:x1855007625282432:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:0.0:1769074933.210403:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625282432:12345-0@lo:2:osp-syn-1-1.0 Request processed in 271us (8065us total) trans 17179874648 rc 0/0 00000100:00100000:0.0:1769074933.210412:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14549 00000100:00100000:0.0:1769074933.210428:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14550 00000100:00100000:0.0:1769074933.210518:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3650:x1855007625282304:12345-0@lo:2:osp-syn-1-1.0 00000100:00100000:0.0:1769074933.210603:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625282304:12345-0@lo:2:osp-syn-1-1.0 Request processed in 169us (1184us total) trans 17179874649 rc 0/0 00000100:00100000:0.0:1769074933.210624:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14550 00000100:00100000:0.0:1769074933.210637:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14551 00000100:00100000:0.0:1769074933.210642:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625282688:12345-0@lo:2:osp-syn-0-1.0 00000100:00100000:0.0:1769074933.210737:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3650:x1855007625282688:12345-0@lo:2:osp-syn-0-1.0 Request processed in 94us (1095us total) trans 21474841314 rc 0/0 00000100:00100000:0.0:1769074933.210743:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14551 00000100:00080000:0.0:1769074933.210763:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074933.211198:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625282560:0@lo:2:osp-syn-0-1.0 00000100:00100000:0.0:1769074933.211233:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074933.211269:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625282560 00000100:00100000:0.0:1769074933.211275:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625282560 00000100:00100000:0.0:1769074933.211390:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14552 00000100:00100000:0.0:1769074933.211397:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3652:x1855007625282560:12345-0@lo:2:osp-syn-0-1.0 00000100:00100000:0.0:1769074933.211579:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625282560:12345-0@lo:2:osp-syn-0-1.0 Request processed in 183us (345us total) trans 21474841315 rc 0/0 00000100:00100000:0.0:1769074933.211585:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14552 00000004:00080000:0.0:1769074933.211713:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d0e4380/2, rc 0, transno 21474841315 00000100:00100000:0.0:1769074933.211719:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625282560:0@lo:2:osp-syn-0-1.0 00000004:00080000:1.0:1769074933.212902:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54c846d80/2, rc 0, transno 17179874648 00000100:00100000:1.0:1769074933.212914:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625282432:0@lo:2:osp-syn-1-1.0 00000004:00080000:2.0:1769074933.214173:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54d0e5c00/2, rc 0, transno 21474841314 00000100:00100000:2.0:1769074933.214185:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625282688:0@lo:2:osp-syn-0-1.0 00000004:00080000:2.0:1769074933.214197:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54c845f80/2, rc 0, transno 17179874649 00000100:00100000:2.0:1769074933.214201:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625282304:0@lo:2:osp-syn-1-1.0 00000100:00100000:1.0:1769074933.227239:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.227315:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617064960 00000100:00100000:2.0:1769074933.227325:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617064960 00000100:00100000:2.0:1769074933.227403:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13666 00000100:00100000:2.0:1769074933.227415:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c847b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617064960:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.227440:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.227461:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca1c expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.227530:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.227547:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.227565:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.227569:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074933.227572:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89400, rc 0) 00000100:00100000:2.0:1769074933.227639:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c847b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617064960:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 227us (412us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.227647:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13666 00000100:00100000:1.0:1769074933.234731:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.234773:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065088 00000100:00100000:1.0:1769074933.234783:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065088 00000100:00100000:1.0:1769074933.234865:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13667 00000100:00100000:1.0:1769074933.234876:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617065088:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.234907:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.234929:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd55f5a lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca23 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.235034:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f61 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074933.235057:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f61 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.235069:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f61 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:1.0:1769074933.235097:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254593 00000004:00200000:1.0:1769074933.235102:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074933.235130:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f61 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.235157:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd55f5a lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca23 expref: 12 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.235175:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f61 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca23 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.235188:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.235191:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074933.235195:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242000, rc 0) 00000100:00100000:1.0:1769074933.235251:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68501:x1855007617065088:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 376us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.235259:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13667 00000100:00100000:1.0:1769074933.242350:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.242385:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.242418:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065216 00000100:00100000:1.0:1769074933.242427:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065216 00000100:00100000:1.0:1769074933.242472:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065344 00000100:00100000:1.0:1769074933.242475:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065344 00000100:00100000:1.0:1769074933.242500:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14553 00000100:00100000:1.0:1769074933.242510:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617065216:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.242528:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.242550:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca2a expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074933.242575:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x36:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074933.242599:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.242601:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.242612:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 9 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074933.242643:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 9 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074933.242662:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243200, rc 0) 00000100:00100000:1.0:1769074933.242735:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617065216:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 211us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.242742:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14553 00000100:00100000:1.0:1769074933.242762:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14554 00000100:00100000:1.0:1769074933.242770:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617065344:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.242778:0:8396:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.242789:0:8396:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca31 expref: -99 pid: 8396 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074933.242829:0:8396:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x38:0x0] initial LVB size: 70254592, mtime: 0x6971f0e7, atime: 0x0, ctime: 0x6971f0e7, blocks: 0x800 00010000:00010000:1.0:1769074933.242841:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.242843:0:8396:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.242849:0:8396:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 10 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074933.242861:0:8396:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 10 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1769074933.242870:0:8396:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243800, rc 0) 00000100:00100000:1.0:1769074933.242892:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617065344:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 123us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.242899:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14554 00000100:00100000:1.0:1769074933.340133:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.340354:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065472 00000100:00100000:0.0:1769074933.340365:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065472 00000100:00100000:0.0:1769074933.340423:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14555 00000100:00100000:0.0:1769074933.340445:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617065472:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074933.340738:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617065472:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 304us (613us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074933.340747:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14555 00000100:00100000:2.0:1769074933.347431:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.348211:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065600 00000100:00100000:0.0:1769074933.348223:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065600 00000100:00100000:0.0:1769074933.348279:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14556 00000100:00100000:0.0:1769074933.348289:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617065600:12345-192.168.204.60@tcp:23:lfs.0 00000100:00100000:0.0:1769074933.348394:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617065600:12345-192.168.204.60@tcp:23:lfs.0 Request processed in 107us (972us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074933.348403:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14556 00000100:00100000:2.0:1769074933.360718:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.363324:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065856 00000100:00100000:1.0:1769074933.363339:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065856 00000100:00100000:1.0:1769074933.363408:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13668 00000100:00100000:1.0:1769074933.363418:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68501:x1855007617065856:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:1.0:1769074933.363481:0:6512:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074933.363508:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f61 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca23 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.363525:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.363530:0:6512:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074933.363723:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074933.363737:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074933.363746:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000001:00080000:1.0:1769074933.363786:0:6512:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 4294968384 00000004:00010000:1.0:1769074933.363823:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000097c501e9 reply state 00000000ccecf6fc transno 4294968385 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074933.363836:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074933.363842:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.363845:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1769074933.363848:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074933.363939:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:68501:x1855007617065856:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 522us (3231us total) trans 4294968385 rc 0/0 00000100:00100000:1.0:1769074933.363947:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13668 00000020:00080000:1.0:1769074933.363952:0:6512:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1769074933.366658:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.366746:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617065984 00000100:00100000:1.0:1769074933.366754:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617065984 00000100:00100000:1.0:1769074933.366835:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14557 00000100:00100000:1.0:1769074933.366846:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617065984:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.366863:0:36516:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.366919:0:36516:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca38 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1769074933.366940:0:36516:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000400:0x37:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1769074933.366958:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.366960:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.366969:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.366986:0:36516:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 11 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.366998:0:36516:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549242200, rc 0) 00000100:00100000:1.0:1769074933.367054:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68501:x1855007617065984:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 210us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.367060:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14557 00000100:00100000:3.0:1769074933.370663:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.370863:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617066112 00000100:00100000:2.0:1769074933.370874:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617066112 00000100:00100000:2.0:1769074933.370932:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14558 00000100:00100000:2.0:1769074933.370941:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68501:x1855007617066112:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.370961:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.370982:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca3f expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074933.371010:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x37:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1769074933.371031:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.371034:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.371043:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.371060:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.371073:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88c00, rc 0) 00000100:00100000:2.0:1769074933.371148:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617066112:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 207us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.371155:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14558 00000100:00100000:3.0:1769074933.376879:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.376956:0:8404:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617066240 00000100:00100000:2.0:1769074933.376966:0:8404:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617066240 00002000:00010000:2.0:1769074933.377007:0:8404:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419264/55 (10485760->18446744073709551615) 00002000:00100000:2.0:1769074933.377013:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5500 with x1855007617066240 ext(10485760->18446744073709551615) 00002000:00010000:2.0:1769074933.377025:0:8404:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef5500. 00000100:00100000:2.0:1769074933.377062:0:8404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8762 00000100:00100000:2.0:1769074933.377071:0:8404:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617066240:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:2.0:1769074933.377257:0:8404:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb size from disk: 0 -> 10485760 00002000:00010000:2.0:1769074933.377262:0:8404:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb mtime from disk: 0 -> 1769074932 00002000:00010000:2.0:1769074933.377264:0:8404:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074933.377266:0:8404:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb ctime from disk: 0 -> 1769074932 00002000:00010000:2.0:1769074933.377269:0:8404:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074933.377328:0:8404:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617066240:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 256us (458us total) trans 21474841316 rc 0/0 00000100:00100000:2.0:1769074933.377335:0:8404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8762 00002000:00010000:2.0:1769074933.377339:0:8404:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419264/55 (10485760->18446744073709551615) 00002000:00100000:2.0:1769074933.377342:0:8404:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef5500 with x1855007617066240 ext(10485760->18446744073709551615) 00002000:00010000:2.0:1769074933.377346:0:8404:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef5500. 00000100:00100000:3.0:1769074933.379860:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.380089:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617066368 00000100:00100000:1.0:1769074933.380099:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617066368 00000100:00100000:1.0:1769074933.380156:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13669 00000100:00100000:1.0:1769074933.380166:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:68501:x1855007617066368:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.380190:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.380210:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f8b lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca46 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.380310:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243e00/0x7fe2323adfd55f92 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:1.0:1769074933.380327:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243e00/0x7fe2323adfd55f92 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074933.380339:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243e00/0x7fe2323adfd55f92 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:1.0:1769074933.380369:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 10485760 00000004:00200000:1.0:1769074933.380373:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074933.380391:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243e00/0x7fe2323adfd55f92 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:1.0:1769074933.380416:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd55f8b lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca46 expref: 15 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.380430:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243e00/0x7fe2323adfd55f92 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca46 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.380439:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.380442:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1769074933.380445:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243e00, rc 0) 00000100:00100000:1.0:1769074933.380506:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:68501:x1855007617066368:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 341us (654us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.380514:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13669 00000100:00100000:3.0:1769074933.386685:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.386730:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617066496 00000100:00100000:3.0:1769074933.386740:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617066496 00000100:00100000:3.0:1769074933.386829:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8763 00000100:00100000:3.0:1769074933.386839:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617066496:12345-192.168.204.60@tcp:3:lfs.0 00010000:00010000:3.0:1769074933.386874:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.386889:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.386892:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769074933.386897:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 3/1,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.386911:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 3/1,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408113:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 3/1,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408130:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408140:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408147:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b67febbe00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:3.0:1769074933.408153:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408162:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408183:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.408191:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b67febbe00/0x7fe2323adfd55f99 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074933.408272:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617066496:12345-192.168.204.60@tcp:3:lfs.0 Request processed in 21430us (21591us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1769074933.408283:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8763 00000100:00100000:3.0:1769074933.416173:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.416209:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617066624 00000100:00100000:3.0:1769074933.416218:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617066624 00000100:00100000:3.0:1769074933.416280:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8764 00000100:00100000:3.0:1769074933.416289:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617066624:12345-192.168.204.60@tcp:4:lfs.0 00010000:00010000:3.0:1769074933.416326:0:28434:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.416347:0:28434:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.416356:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074933.416359:0:28434:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074933.416364:0:28434:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.416398:0:28434:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.416413:0:28434:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 11 pid: 8398 timeout: 2021 lvb_type: 0 00000100:00100000:3.0:1769074933.416796:0:28434:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_010:lustre-OST0000_UUID:28434:1855007625282816:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074933.416893:0:28434:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b492540 going to sleep for 11 seconds 00000100:00100000:3.0:1769074933.419387:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.419458:0:28434:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_010:lustre-OST0000_UUID:28434:1855007625282816:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074933.419484:0:28434:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074933.421359:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617066752 00000100:00100000:1.0:1769074933.421372:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617066752 00010000:00010000:1.0:1769074933.421410:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 12 pid: 8398 timeout: 2021 lvb_type: 0 00000100:00100000:1.0:1769074933.421458:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7236 00000100:00100000:1.0:1769074933.421468:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2408:x1855007617066752:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769074933.421483:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074933.421502:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 12 pid: 8398 timeout: 2021 lvb_type: 0 00010000:00010000:1.0:1769074933.421517:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 12 pid: 8398 timeout: 2021 lvb_type: 0 00010000:00010000:1.0:1769074933.421540:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55f84 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca3f expref: 12 pid: 8398 timeout: 2021 lvb_type: 0 00010000:00010000:1.0:1769074933.421555:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.421558:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769074933.421562:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.421572:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.421601:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769074933.421633:0:28434:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.421643:0:28434:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074933.421669:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617066752:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 201us (2289us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.421677:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7236 00000001:00080000:0.0:1769074933.460310:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841317 is committed 00000001:00080000:0.0:1769074933.460334:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841315 is committed 00002000:00010000:1.0:1769074933.460529:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb size from disk: 10485760 -> 70254592 00002000:00010000:1.0:1769074933.460534:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb mtime from disk: 1769074932 -> 1769074933 00002000:00010000:1.0:1769074933.460537:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb ctime from disk: 1769074932 -> 1769074933 00002000:00010000:1.0:1769074933.460540:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb blocks from disk: 0 -> 2048 00010000:00010000:1.0:1769074933.460554:0:28434:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.460566:0:28434:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.460575:0:28434:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.460582:0:28434:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549278600 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074933.460587:0:28434:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.460595:0:28434:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.460625:0:28434:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.460633:0:28434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549278600/0x7fe2323adfd55fa0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 28434 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1769074933.460734:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617066624:12345-192.168.204.60@tcp:4:lfs.0 Request processed in 44442us (44569us total) trans 21474841317 rc 0/0 00000100:00100000:1.0:1769074933.460745:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8764 00000100:00100000:3.0:1769074933.482328:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.482409:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067008 00000100:00100000:1.0:1769074933.482429:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067008 00000100:00100000:1.0:1769074933.482500:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13670 00000100:00100000:1.0:1769074933.482510:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+14:68501:x1855007617067008:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:1.0:1769074933.482566:0:10283:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769074933.482592:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243e00/0x7fe2323adfd55f92 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca46 expref: 14 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.482606:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.482621:0:10283:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074933.482788:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074933.482803:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.482840:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000001:00080000:1.0:1769074933.482884:0:10283:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 4294968386 00000004:00010000:1.0:1769074933.482907:0:10283:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 0000000051fb0858 reply state 00000000268bb24a transno 4294968387 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.482918:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.482924:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.482927:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1769074933.482929:0:10283:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074933.483008:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:68501:x1855007617067008:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 499us (688us total) trans 4294968387 rc 0/0 00000100:00100000:1.0:1769074933.483016:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13670 00000020:00080000:1.0:1769074933.483021:0:10283:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1769074933.486766:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.487067:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067136 00000100:00100000:2.0:1769074933.487078:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067136 00000100:00100000:2.0:1769074933.487130:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14559 00000100:00100000:2.0:1769074933.487141:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68501:x1855007617067136:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.487158:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.487180:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 2/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca4d expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1769074933.487202:0:8398:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x37:0x0] initial LVB size: 70254592, mtime: 0x6971f0f5, atime: 0x0, ctime: 0x6971f0f5, blocks: 0x800 00010000:00010000:2.0:1769074933.487220:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.487223:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.487231:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.487248:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 10 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.487265:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88c00, rc 0) 00000100:00100000:2.0:1769074933.487321:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617067136:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 182us (565us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.487330:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14559 00000100:00100000:3.0:1769074933.491922:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.491980:0:28434:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067264 00000100:00100000:1.0:1769074933.491992:0:28434:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067264 00002000:00010000:1.0:1769074933.492029:0:28434:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419264/55 (70254592->18446744073709551615) 00002000:00100000:1.0:1769074933.492033:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef6300 with x1855007617067264 ext(70254592->18446744073709551615) 00002000:00010000:1.0:1769074933.492044:0:28434:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef6300. 00000100:00100000:1.0:1769074933.492069:0:28434:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 8765 00000100:00100000:1.0:1769074933.492089:0:28434:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617067264:12345-192.168.204.60@tcp:10:lfs.0 00002000:00010000:1.0:1769074933.492246:0:28434:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb size from disk: 70254592 -> 70254592 00002000:00010000:1.0:1769074933.492250:0:28434:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb mtime from disk: 1769074933 -> 1769074933 00002000:00010000:1.0:1769074933.492252:0:28434:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1769074933.492254:0:28434:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb ctime from disk: 1769074933 -> 1769074933 00002000:00010000:1.0:1769074933.492257:0:28434:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:1.0:1769074933.492309:0:28434:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617067264:12345-192.168.204.60@tcp:10:lfs.0 Request processed in 231us (395us total) trans 21474841318 rc 0/0 00000100:00100000:1.0:1769074933.492317:0:28434:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 8765 00002000:00010000:1.0:1769074933.492321:0:28434:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419264/55 (70254592->18446744073709551615) 00002000:00100000:1.0:1769074933.492324:0:28434:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff89b547ef6300 with x1855007617067264 ext(70254592->18446744073709551615) 00002000:00010000:1.0:1769074933.492328:0:28434:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff89b547ef6300. 00000100:00100000:3.0:1769074933.500629:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.500880:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067520 00000100:00100000:2.0:1769074933.500891:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067520 00000100:00100000:2.0:1769074933.501532:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13671 00000100:00100000:2.0:1769074933.502042:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+16:68501:x1855007617067520:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074933.502212:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.502231:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.502244:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00010000:2.0:1769074933.502348:0:6512:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000e55213d4 reply state 000000001095a2ec transno 4294968388 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.502360:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.502368:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.502371:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074933.502375:0:6512:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1769074933.502475:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:68501:x1855007617067520:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 930us (1867us total) trans 4294968388 rc 0/0 00000100:00100000:2.0:1769074933.502483:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13671 00000100:00100000:3.0:1769074933.511867:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.512354:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067648 00000100:00100000:2.0:1769074933.512364:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067648 00000100:00100000:2.0:1769074933.512420:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14560 00000100:00100000:2.0:1769074933.512429:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68501:x1855007617067648:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.512446:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.512481:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 2/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca54 expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.512511:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074933.512519:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.512521:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.512525:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->67108863] (req 0->67108863) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.512557:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074933.512569:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 12 pid: 8396 timeout: 2021 lvb_type: 1 00000100:00100000:2.0:1769074933.512582:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625282944:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074933.512666:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b462840 going to sleep for 11 seconds 00000100:00100000:0.0:1769074933.528178:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.528210:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067776 00000100:00100000:0.0:1769074933.528220:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067776 00010000:00010000:0.0:1769074933.528256:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 13 pid: 8396 timeout: 2021 lvb_type: 1 00000100:00100000:2.0:1769074933.529437:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8398:1855007625282944:192.168.204.60@tcp:104: 00000100:00100000:0.0:1769074933.529647:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7237 00000100:00100000:0.0:1769074933.529661:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+13:2409:x1855007617067776:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074933.529678:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074933.529700:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 13 pid: 8396 timeout: 2021 lvb_type: 1 00010000:00010000:0.0:1769074933.529714:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 13 pid: 8396 timeout: 2021 lvb_type: 1 00010000:00010000:0.0:1769074933.529737:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243200/0x7fe2323adfd55f68 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca2a expref: 13 pid: 8396 timeout: 2021 lvb_type: 1 00010000:00010000:0.0:1769074933.529751:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074933.529753:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074933.529760:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.529768:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.529802:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074933.532092:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54920a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625283072:192.168.204.60@tcp:105: 00000100:00100000:0.0:1769074933.532191:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266780 going to sleep for 11 seconds 00010000:00010000:2.0:1769074933.532410:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.532435:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88e00, rc 0) 00000100:00100000:2.0:1769074933.532476:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68501:x1855007617067648:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 20047us (20617us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.532485:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14560 00000100:00100000:2.0:1769074933.537749:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54920a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6500:1855007625283072:192.168.204.60@tcp:105: 00010000:00010000:2.0:1769074933.537775:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074933.548076:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d0e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617067776:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 18408us (19897us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.548091:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7237 00000100:00100000:0.0:1769074933.550801:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.551746:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617067904 00000100:00100000:2.0:1769074933.551758:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617067904 00000100:00100000:2.0:1769074933.551826:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14561 00000100:00100000:2.0:1769074933.551836:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:68501:x1855007617067904:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.551853:0:8398:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.551871:0:8398:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca5b expref: -99 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.551902:0:8398:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074933.551909:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.551911:0:8398:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.551915:0:8398:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 4/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [67108864->134217727] (req 67108864->134217727) gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.553077:0:8398:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1769074933.553094:0:8398:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 2021 lvb_type: 1 00000100:00100000:2.0:1769074933.553110:0:8398:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625283200:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074933.553209:0:8398:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b462840 going to sleep for 11 seconds 00000100:00100000:2.0:1769074933.565104:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.565177:0:8398:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8398:1855007625283200:192.168.204.60@tcp:104: 00010000:00010000:2.0:1769074933.565208:0:8398:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [67108864->134217727] (req 67108864->134217727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 12 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.565228:0:8398:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88000, rc 0) 00000100:00100000:2.0:1769074933.565286:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68501:x1855007617067904:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 13450us (14493us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.565297:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14561 00000100:00100000:2.0:1769074933.565335:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617068032 00000100:00100000:2.0:1769074933.565341:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617068032 00010000:00010000:2.0:1769074933.565365:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 2021 lvb_type: 1 00000100:00100000:2.0:1769074933.565391:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7238 00000100:00100000:2.0:1769074933.565400:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617068032:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074933.565416:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074933.565436:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 2021 lvb_type: 1 00010000:00010000:2.0:1769074933.565448:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 2021 lvb_type: 1 00010000:00010000:2.0:1769074933.565470:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549243800/0x7fe2323adfd55f6f lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca31 expref: 12 pid: 8396 timeout: 2021 lvb_type: 1 00010000:00010000:2.0:1769074933.565483:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.565485:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074933.565491:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.565500:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40080000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.565529:0:6500:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 11 pid: 8398 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074933.565543:0:6500:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625283328:192.168.204.60@tcp:105: 00000100:00100000:2.0:1769074933.565590:0:6500:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b462840 going to sleep for 11 seconds 00000100:00100000:3.0:1769074933.576093:0:6500:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6500:1855007625283328:192.168.204.60@tcp:105: 00010000:00010000:3.0:1769074933.576120:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074933.579074:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617068032:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 13663us (13968us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.579088:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7238 00000100:00100000:0.0:1769074933.583130:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.583212:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.583239:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617068160 00000100:00100000:0.0:1769074933.583249:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617068160 00000100:00100000:0.0:1769074933.583291:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617068288 00000100:00100000:0.0:1769074933.583294:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617068288 00000100:00100000:0.0:1769074933.583316:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14562 00000100:00100000:0.0:1769074933.583324:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617068160:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074933.583477:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1769074933.583480:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb mtime from disk: 0 -> 1769074919 00002000:00010000:0.0:1769074933.583483:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb atime from disk: 0 -> 1769074907 00002000:00010000:0.0:1769074933.583485:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb ctime from disk: 0 -> 1769074933 00002000:00010000:0.0:1769074933.583487:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1769074933.583538:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617068160:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 215us (417us total) trans 21474841319 rc 0/0 00000100:00100000:0.0:1769074933.583545:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14562 00000100:00100000:0.0:1769074933.583563:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 14563 00000100:00100000:0.0:1769074933.583570:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2409:x1855007617068288:12345-192.168.204.60@tcp:2:lfs.0 00002000:00010000:0.0:1769074933.583649:0:36516:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb size from disk: 70254592 -> 70254592 00002000:00010000:0.0:1769074933.583652:0:36516:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb mtime from disk: 1769074919 -> 1769074919 00002000:00010000:0.0:1769074933.583654:0:36516:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb atime from disk: 0 -> 1769074907 00002000:00010000:0.0:1769074933.583656:0:36516:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb ctime from disk: 1769074919 -> 1769074933 00002000:00010000:0.0:1769074933.583658:0:36516:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:0.0:1769074933.583677:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2409:x1855007617068288:12345-192.168.204.60@tcp:2:lfs.0 Request processed in 108us (466us total) trans 17179874650 rc 0/0 00000100:00100000:0.0:1769074933.583684:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 14563 00000100:00100000:0.0:1769074933.589163:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.589656:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617068544 00000100:00100000:2.0:1769074933.589667:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617068544 00000100:00100000:2.0:1769074933.589735:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4753 00000100:00100000:2.0:1769074933.589745:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+18:68501:x1855007617068544:12345-192.168.204.60@tcp:35:lfs.0 00000004:00010000:2.0:1769074933.589834:0:15947:0:(mdt_open.c:2499:mdt_close_resync_done()) ### [0x240000404:0x4d:0x0] lease broken? 0 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55f30 lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca15 expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.589899:0:15947:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 15947 initiator: MDT0 00010000:00010000:2.0:1769074933.589913:0:15947:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 2/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.589923:0:15947:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 4/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.589957:0:15947:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 18 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.589969:0:15947:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 4/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 18 pid: 6512 timeout: 2021 lvb_type: 0 00000100:00100000:2.0:1769074933.589984:0:15947:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0001_UUID:15947:1855007625283456:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074933.590063:0:15947:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b462600 going to sleep for 11 seconds 00000100:00100000:2.0:1769074933.594467:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.594554:0:15947:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0001_UUID:15947:1855007625283456:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074933.594582:0:15947:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00000100:00100000:3.0:1769074933.594655:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617068672 00000100:00100000:3.0:1769074933.594664:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617068672 00010000:00010000:3.0:1769074933.594702:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 4/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 19 pid: 6512 timeout: 2021 lvb_type: 0 00000100:00100000:3.0:1769074933.594735:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7239 00000100:00100000:3.0:1769074933.594744:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007617068672:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769074933.594763:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769074933.594770:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 4/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 19 pid: 6512 timeout: 2021 lvb_type: 0 00010000:00010000:3.0:1769074933.594783:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 19 pid: 6512 timeout: 2021 lvb_type: 0 00010000:00010000:3.0:1769074933.594841:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89400/0x7fe2323adfd55f53 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 9 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca1c expref: 19 pid: 6512 timeout: 2021 lvb_type: 0 00010000:00010000:3.0:1769074933.594858:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074933.594862:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00010000:00010000:3.0:1769074933.594872:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 15947 initiator: MDT1 00010000:00010000:3.0:1769074933.594884:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 4/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 15947 initiator: MDT1 00010000:00010000:3.0:1769074933.594909:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074933.595027:0:15947:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00010000:00010000:0.0:1769074933.595034:0:15947:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00000040:00080000:0.0:1769074933.595272:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].70, 88 off13800 00000004:00080000:0.0:1769074933.595280:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1921437472971ns, next at 1833854042001ns, rc = 0 00000040:00080000:0.0:1769074933.595314:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].75, 88 off14144 00000004:00080000:0.0:1769074933.595319:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1921437511691ns, next at 1835465451293ns, rc = 0 00000040:00080000:0.0:1769074933.595336:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].76, 88 off14232 00000004:00080000:0.0:1769074933.595340:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1921437532930ns, next at 1835465451293ns, rc = 0 00000040:00080000:0.0:1769074933.595349:0:15947:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].71, 88 off13888 00000004:00080000:0.0:1769074933.595353:0:15947:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1921437545806ns, next at 1833854042001ns, rc = 0 00000100:00100000:3.0:1769074933.595683:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+19:2409:x1855007617068672:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 937us (1224us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.595691:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7239 00000001:00080000:2.0:1769074933.599704:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968389 is committed 00000004:00080000:2.0:1769074933.599718:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 70 committed 00000004:00080000:2.0:1769074933.599765:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 75 committed 00000004:00080000:2.0:1769074933.599784:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 74 committed 00000004:00080000:2.0:1769074933.599788:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 69 committed 00000100:00100000:3.0:1769074933.599886:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1769074933.599905:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625283840:0@lo:2:osp-syn-0-1.0 00000100:00100000:3.0:1769074933.599905:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625283584:0@lo:2:osp-syn-1-1.0 00010000:00010000:0.0:1769074933.600054:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.600065:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.600071:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.600077:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.600082:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.600089:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00000004:00010000:2.0:1769074933.600297:0:15947:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000ea0e9f57 reply state 000000003470bd40 transno 4294968389 ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.600371:0:15947:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.600380:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.600383:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00000100:00100000:2.0:1769074933.600387:0:15947:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074933.600394:0:15947:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.600396:0:15947:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769074933.600402:0:15947:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd55f30 lrc: 0/0,0 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x4/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca15 expref: 12 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074933.600489:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074933.600492:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074933.600496:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00080000:2.0:1769074933.600500:0:15947:0:(ldlm_lib.c:3435:target_send_reply()) Schedule reply immediately 00010000:00010000:0.0:1769074933.600503:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243000/0x7fe2323adfd55f76 lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:2.0:1769074933.600512:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:68501:x1855007617068544:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 10769us (11359us total) trans 4294968389 rc 0/0 00010000:00010000:0.0:1769074933.600517:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.600522:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00000100:00100000:2.0:1769074933.600522:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4753 00010000:00010000:0.0:1769074933.600527:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.600532:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.600535:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.600539:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.600547:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074933.600549:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074933.600552:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.600561:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242800/0x7fe2323adfd55fa7 lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:3.0:1769074933.600602:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074933.600664:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625283712:0@lo:2:osp-syn-0-1.0 00000100:00100000:3.0:1769074933.600686:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074933.600718:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625283968:0@lo:2:osp-syn-1-1.0 00000100:00100000:3.0:1769074933.600740:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074933.600771:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625283584 00000100:00100000:3.0:1769074933.600779:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625283584 00000100:00100000:3.0:1769074933.600830:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625283712 00000100:00100000:3.0:1769074933.600834:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625283712 00000100:00100000:3.0:1769074933.600852:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625283968 00000100:00100000:3.0:1769074933.600855:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625283968 00000100:00100000:3.0:1769074933.600873:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14564 00000100:00100000:3.0:1769074933.600880:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+8:3649:x1855007625283584:12345-0@lo:2:osp-syn-1-1.0 00010000:00010000:2.0:1769074933.601285:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.601293:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.601298:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.601304:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.601307:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.601313:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.601323:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.601326:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769074933.601329:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT1 00010000:00010000:2.0:1769074933.601335:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd55fca lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 15947 initiator: MDT1 00002000:00010000:3.0:1769074933.601466:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb size from disk: 70254592 -> 70254592 00002000:00010000:3.0:1769074933.601470:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb mtime from disk: 1769074919 -> 1769074919 00002000:00010000:3.0:1769074933.601472:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb atime from disk: 1769074907 -> 1769074907 00002000:00010000:3.0:1769074933.601474:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb ctime from disk: 1769074933 -> 1769074933 00002000:00010000:3.0:1769074933.601477:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x38:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:3.0:1769074933.601518:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625283584:12345-0@lo:2:osp-syn-1-1.0 Request processed in 638us (917us total) trans 17179874651 rc 0/0 00000100:00100000:3.0:1769074933.601525:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14564 00000100:00100000:3.0:1769074933.601543:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14565 00000100:00100000:3.0:1769074933.601551:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625283712:12345-0@lo:2:osp-syn-0-1.0 00000100:00100000:1.0:1769074933.601834:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074933.601936:0:27844:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625283840 00000100:00100000:2.0:1769074933.601943:0:27844:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625283840 00000100:00100000:2.0:1769074933.601979:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14566 00000100:00100000:2.0:1769074933.601986:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+8:3652:x1855007625283968:12345-0@lo:2:osp-syn-1-1.0 00002000:00010000:2.0:1769074933.602044:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x2c0000400:0x37:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1769074933.602047:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x2c0000400:0x37:0x0] updating lvb mtime from disk: 0 -> 0 00002000:00010000:2.0:1769074933.602049:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x2c0000400:0x37:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074933.602051:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x2c0000400:0x37:0x0] updating lvb ctime from disk: 0 -> 0 00002000:00010000:2.0:1769074933.602053:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x2c0000400:0x37:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1769074933.602086:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+9:3652:x1855007625283968:12345-0@lo:2:osp-syn-1-1.0 Request processed in 101us (1344us total) trans 17179874652 rc 0/0 00000100:00100000:2.0:1769074933.602092:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14566 00000100:00100000:2.0:1769074933.602117:0:27844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14567 00000100:00100000:2.0:1769074933.602123:0:27844:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625283840:12345-0@lo:2:osp-syn-0-1.0 00002000:00010000:3.0:1769074933.602147:0:8396:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1769074933.602151:0:8396:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb mtime from disk: 1769074919 -> 1769074919 00002000:00010000:3.0:1769074933.602154:0:8396:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb atime from disk: 1769074907 -> 1769074907 00002000:00010000:3.0:1769074933.602156:0:8396:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb ctime from disk: 1769074933 -> 1769074933 00002000:00010000:3.0:1769074933.602158:0:8396:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x36:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:2.0:1769074933.602171:0:27844:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb size from disk: 70254592 -> 70254592 00002000:00010000:2.0:1769074933.602173:0:27844:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb mtime from disk: 1769074933 -> 1769074933 00002000:00010000:2.0:1769074933.602176:0:27844:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1769074933.602178:0:27844:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb ctime from disk: 1769074933 -> 1769074933 00002000:00010000:2.0:1769074933.602180:0:27844:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x37:0x0] updating lvb blocks from disk: 2048 -> 2048 00000100:00100000:3.0:1769074933.602183:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+8:3649:x1855007625283712:12345-0@lo:2:osp-syn-0-1.0 Request processed in 634us (1495us total) trans 21474841320 rc 0/0 00000100:00100000:3.0:1769074933.602188:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14565 00000100:00100000:2.0:1769074933.602203:0:27844:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625283840:12345-0@lo:2:osp-syn-0-1.0 Request processed in 80us (368us total) trans 21474841321 rc 0/0 00000100:00100000:2.0:1769074933.602209:0:27844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14567 00000004:00080000:3.0:1769074933.602243:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ece00/2, rc 0, transno 17179874651 00000100:00100000:3.0:1769074933.602248:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625283584:0@lo:2:osp-syn-1-1.0 00000100:00100000:1.0:1769074933.602284:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 21474841315 gen 2 00000004:00080000:1.0:1769074933.602288:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d0e5c00, transno 21474841314 00000004:00080000:1.0:1769074933.602294:0:3651:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54d0e4380, transno 21474841315 00000100:00100000:1.0:1769074933.602301:0:3651:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff89b54441fb80 x1855007625283840/t21474841321(21474841321) o2->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 440/432 e 0 to 0 dl 1769074949 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-1.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1769074933.602302:0:3636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.602311:0:3636:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 2/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00000004:00080000:1.0:1769074933.602317:0:3651:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b54441fb80/2, rc 0, transno 21474841321 00010000:00010000:0.0:1769074933.602318:0:3636:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00000100:00100000:1.0:1769074933.602321:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625283840:0@lo:2:osp-syn-0-1.0 00010000:00010000:0.0:1769074933.602323:0:3636:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.602327:0:3636:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.602333:0:3636:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 4/0,0 mode: TXN/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.602343:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074933.602345:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074933.602348:0:3636:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 2/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074933.602353:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd55fb5 lrc: 0/0,0 mode: --/TXN res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000004:00080000:2.0:1769074933.603665:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491edf80/2, rc 0, transno 17179874652 00000100:00100000:2.0:1769074933.603676:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625283968:0@lo:2:osp-syn-1-1.0 00000004:00080000:3.0:1769074933.603684:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546c68a80/2, rc 0, transno 21474841320 00000100:00100000:3.0:1769074933.603758:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625283712:0@lo:2:osp-syn-0-1.0 00000100:00100000:2.0:1769074933.604492:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.604561:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617068800 00000100:00100000:1.0:1769074933.604569:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617068800 00000100:00100000:1.0:1769074933.604631:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13672 00000100:00100000:1.0:1769074933.604641:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:68501:x1855007617068800:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:1.0:1769074933.604663:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769074933.604682:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55fd1 lrc: 2/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca62 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.604746:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55fd1 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca62 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.604759:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55fd1 lrc: 3/0,0 mode: CR/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca62 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074933.604775:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.604778:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1769074933.604782:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549243600, rc 0) 00000100:00100000:1.0:1769074933.604862:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68501:x1855007617068800:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 222us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.604874:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13672 00000100:00100000:2.0:1769074933.624900:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.625155:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617069056 00000100:00100000:0.0:1769074933.625168:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617069056 00000100:00100000:0.0:1769074933.625231:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4754 00000100:00100000:0.0:1769074933.625241:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68501:x1855007617069056:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:0.0:1769074933.626045:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68501:x1855007617069056:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 801us (1155us total) trans 4294968390 rc 0/0 00000100:00100000:0.0:1769074933.626056:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4754 00000100:00100000:1.0:1769074933.660776:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074933.661289:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625284096:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074933.661348:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074933.661450:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284096 00000100:00100000:3.0:1769074933.661460:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284096 00000100:00100000:3.0:1769074933.661517:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1295 00000100:00100000:3.0:1769074933.661527:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625284096:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074933.661550:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 60273984 pending 1073152 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074933.661710:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625284096:12345-0@lo:13:osp-pre-1-1.0 Request processed in 184us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.661723:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1295 00000100:00100000:0.0:1769074933.661824:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625284096:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074933.662787:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617069312 00000100:00100000:2.0:1769074933.662797:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617069312 00000100:00100000:3.0:1769074933.663128:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13673 00000100:00100000:3.0:1769074933.663139:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68502:x1855007617069312:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074933.663159:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074933.663182:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fd8 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca69 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.663259:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.663273:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663284:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663296:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.663302:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663308:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663391:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663397:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663403:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663409:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663415:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.663419:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663425:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663435:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ead00) 00010000:00010000:3.0:1769074933.663438:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.663442:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663447:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55fdf lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663459:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663465:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663470:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663475:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.663477:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663483:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663490:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea900) 00010000:00010000:3.0:1769074933.663492:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.663495:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663500:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55fe6 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.663533:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fd8 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca69 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.663542:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ead00) 00010000:00010000:3.0:1769074933.663544:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074933.663548:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fd8 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca69 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.663558:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549278200, rc 0) 00000100:00100000:3.0:1769074933.664058:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68502:x1855007617069312:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 919us (3291us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074933.664066:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13673 00000100:00100000:0.0:1769074933.670478:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.670601:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617069440 00000100:00100000:3.0:1769074933.670619:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617069440 00000100:00100000:3.0:1769074933.670684:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13674 00000100:00100000:3.0:1769074933.670693:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68502:x1855007617069440:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074933.670773:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.670788:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.670800:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.670844:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.670850:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.670856:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671237:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.671248:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671255:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:3.0:1769074933.671295:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:3.0:1769074933.671300:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074933.671311:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671324:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671331:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671336:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671342:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671352:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549278200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.671357:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671364:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671375:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ead00) 00010000:00010000:3.0:1769074933.671378:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.671383:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671388:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd55fed lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671404:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671408:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671413:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671418:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.671420:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671426:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671433:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea900) 00010000:00010000:3.0:1769074933.671435:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.671438:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671442:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd55ff4 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671452:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671456:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671461:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671465:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.671467:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671471:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671477:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074933.671479:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.671481:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.671487:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd55ffb lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:3.0:1769074933.671622:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68502:x1855007617069440:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 928us (1149us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.671631:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13674 00000100:00100000:2.0:1769074933.720774:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.725328:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617069696 00000100:00100000:3.0:1769074933.725347:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617069696 00000100:00100000:3.0:1769074933.725706:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13675 00000100:00100000:3.0:1769074933.725718:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68507:x1855007617069696:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074933.725740:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074933.725765:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd56002 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca70 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.725875:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.725902:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.725913:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.725928:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.725934:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.725940:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726305:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726313:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726319:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726329:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726335:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.726339:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726347:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726362:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea900) 00010000:00010000:3.0:1769074933.726365:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.726370:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726376:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd56009 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726388:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726395:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726401:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726405:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549278200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.726408:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726413:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726577:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ead00) 00010000:00010000:3.0:1769074933.726581:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.726585:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726591:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56010 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.726640:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd56002 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca70 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.726650:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea900) 00010000:00010000:3.0:1769074933.726652:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074933.726656:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd56002 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca70 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.726667:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549279c00, rc 0) 00000100:00100000:3.0:1769074933.726730:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68507:x1855007617069696:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1015us (5966us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074933.730178:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13675 00000100:00100000:1.0:1769074933.736631:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.736771:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617069824 00000100:00100000:2.0:1769074933.736781:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617069824 00000100:00100000:2.0:1769074933.736873:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13676 00000100:00100000:2.0:1769074933.737355:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68507:x1855007617069824:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074933.737686:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.737705:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.737717:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.737731:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.737738:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.737744:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738082:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.738092:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738192:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:2.0:1769074933.738234:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:2.0:1769074933.738238:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074933.738251:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738356:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738364:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738370:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738376:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738382:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.738387:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738393:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738405:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074933.738408:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.738413:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738419:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56017 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738590:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738597:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738602:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738616:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.738619:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738624:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738632:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa200) 00010000:00010000:2.0:1769074933.738634:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.738637:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738642:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5601e lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738654:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738658:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738663:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738667:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.738670:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738675:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738682:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.738684:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.738686:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.738692:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56025 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:2.0:1769074933.738788:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68507:x1855007617069824:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1903us (2163us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.738799:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13676 00000100:00100000:0.0:1769074933.793746:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.794867:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617070080 00000100:00100000:2.0:1769074933.794959:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617070080 00000100:00100000:2.0:1769074933.795087:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13677 00000100:00100000:2.0:1769074933.795108:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68515:x1855007617070080:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.795128:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.795152:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5602c lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca77 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.796670:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.796689:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796701:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796773:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.796778:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796783:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796861:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796866:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796870:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796874:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796878:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.796944:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796952:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796963:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074933.796966:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.796971:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796976:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56033 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.796990:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797151:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797156:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797159:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.797161:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797164:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797170:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa800) 00010000:00010000:2.0:1769074933.797172:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.797174:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797177:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5603a lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.797211:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5602c lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca77 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.797226:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074933.797227:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074933.797230:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd5602c lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca77 expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.797242:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b88800, rc 0) 00000100:00100000:2.0:1769074933.800936:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68515:x1855007617070080:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 5828us (7188us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074933.800949:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13677 00000100:00100000:0.0:1769074933.807888:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.808680:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617070208 00000100:00100000:2.0:1769074933.808692:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617070208 00000100:00100000:2.0:1769074933.808758:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13678 00000100:00100000:2.0:1769074933.808767:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68515:x1855007617070208:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:2.0:1769074933.808899:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.808914:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.808925:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.808938:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.808945:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.808951:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809031:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.809041:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809048:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:2.0:1769074933.809087:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:2.0:1769074933.809091:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074933.809104:0:6512:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809118:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809125:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809130:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809137:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809143:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.809146:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809153:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809164:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa200) 00010000:00010000:2.0:1769074933.809167:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.809172:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809178:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56041 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809198:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809202:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809207:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809212:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.809215:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809220:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809227:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa800) 00010000:00010000:2.0:1769074933.809230:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.809232:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809237:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd56048 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809249:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809254:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809561:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809569:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.809574:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809580:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809589:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:2.0:1769074933.809592:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.809594:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.809599:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd5604f lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00000100:00100000:2.0:1769074933.809680:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68515:x1855007617070208:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 913us (1802us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074933.809688:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13678 00000100:00100000:1.0:1769074933.864924:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074933.865291:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617070464 00000100:00100000:2.0:1769074933.865302:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617070464 00000100:00100000:2.0:1769074933.865370:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13679 00000100:00100000:2.0:1769074933.865380:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68520:x1855007617070464:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074933.865402:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074933.865439:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56056 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca7e expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.865517:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.865533:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865543:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865556:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074933.865561:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865566:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865651:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865659:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865665:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865671:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865676:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.865680:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865687:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865698:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074933.865702:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.865706:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865712:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89800/0x7fe2323adfd5605d lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865726:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865731:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865735:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865740:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074933.865742:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865747:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865752:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b64effa200) 00010000:00010000:2.0:1769074933.865754:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074933.865756:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865761:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b88800/0x7fe2323adfd56064 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074933.865792:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56056 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca7e expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.865802:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b64effa800) 00010000:00010000:2.0:1769074933.865804:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1769074933.866250:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b666b89a00/0x7fe2323adfd56056 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca7e expref: 11 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074933.866270:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89a00, rc 0) 00000100:00100000:2.0:1769074933.866349:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68520:x1855007617070464:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 970us (1433us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074933.866357:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13679 00000100:00100000:0.0:1769074933.874759:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074933.875167:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617070592 00000100:00100000:1.0:1769074933.875180:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617070592 00000100:00100000:1.0:1769074933.875249:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13680 00000100:00100000:1.0:1769074933.875259:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68520:x1855007617070592:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:1.0:1769074933.875337:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074933.875353:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875364:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875378:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074933.875384:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875390:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875470:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:1.0:1769074933.875479:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875485:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:1.0:1769074933.875523:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:1.0:1769074933.875528:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1769074933.875539:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875556:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875562:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875567:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875573:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875578:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074933.875582:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875588:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875600:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b681697300) 00010000:00010000:1.0:1769074933.875613:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074933.875620:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875626:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242600/0x7fe2323adfd5606b lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875643:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875649:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875654:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875659:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074933.875661:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875666:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875673:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b681697700) 00010000:00010000:1.0:1769074933.875676:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074933.875678:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875682:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243c00/0x7fe2323adfd56072 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875692:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875697:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875702:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875707:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074933.875710:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875716:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875721:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074933.875724:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074933.875727:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:1.0:1769074933.875732:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243400/0x7fe2323adfd56079 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:1.0:1769074933.877281:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68520:x1855007617070592:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 2013us (2525us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074933.877294:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13680 00000100:00100000:1.0:1769074933.904943:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.905268:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617070848 00000100:00100000:3.0:1769074933.905280:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617070848 00000100:00100000:3.0:1769074933.905402:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13681 00000100:00100000:3.0:1769074933.905412:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68529:x1855007617070848:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074933.905441:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074933.905465:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd56080 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca85 expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.905545:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.905559:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905570:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905582:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.905588:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905595:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905676:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905682:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905688:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905694:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905699:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549278200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.905703:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905714:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905735:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea900) 00010000:00010000:3.0:1769074933.905738:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.905744:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905750:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56087 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905764:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905769:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905775:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905779:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.905781:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905786:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905792:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ead00) 00010000:00010000:3.0:1769074933.905794:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.905797:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905802:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd5608e lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.905887:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd56080 lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca85 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.905897:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea900) 00010000:00010000:3.0:1769074933.905899:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074933.905903:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd56080 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca85 expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.905913:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549279c00, rc 0) 00000100:00100000:3.0:1769074933.905993:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68529:x1855007617070848:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 582us (1060us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074933.906001:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13681 00000100:00100000:0.0:1769074933.917558:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.917642:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617070976 00000100:00100000:3.0:1769074933.917653:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617070976 00000100:00100000:3.0:1769074933.917714:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13682 00000100:00100000:3.0:1769074933.917724:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68529:x1855007617070976:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074933.917804:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.917944:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.917956:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.917970:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.917976:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.917983:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918053:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.918061:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918068:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:3.0:1769074933.918107:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:3.0:1769074933.918111:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074933.918122:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918135:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918141:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918146:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918153:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918158:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549278200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.918162:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918169:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.918181:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543eab00) 00010000:00010000:3.0:1769074933.918184:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.919325:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919336:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd56095 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919360:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919365:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919371:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919377:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.919379:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919385:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919397:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea300) 00010000:00010000:3.0:1769074933.919400:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.919403:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919408:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd5609c lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919420:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919427:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919432:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919437:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.919440:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919445:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919452:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074933.919454:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.919457:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.919462:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560a3 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:3.0:1769074933.919566:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68529:x1855007617070976:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 1838us (2015us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074933.919576:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13682 00000100:00100000:1.0:1769074933.960425:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.960565:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617071232 00000100:00100000:3.0:1769074933.960577:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617071232 00000100:00100000:3.0:1769074933.960646:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13683 00000100:00100000:3.0:1769074933.960656:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68531:x1855007617071232:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074933.960675:0:10283:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074933.960698:0:10283:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560aa lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca8c expref: -99 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.960776:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.960789:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.960800:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.960993:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.961002:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961008:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961078:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961085:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961090:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961096:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961102:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.961106:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961117:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961126:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:3.0:1769074933.961129:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.961134:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961139:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279400/0x7fe2323adfd560b1 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961150:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961155:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961159:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961163:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549278200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074933.961165:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961170:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961176:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543eab00) 00010000:00010000:3.0:1769074933.961178:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074933.961181:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961263:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560b8 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.961298:0:10283:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560aa lrc: 1/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca8c expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.961307:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea300) 00010000:00010000:3.0:1769074933.961310:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074933.961313:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560aa lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca8c expref: 11 pid: 10283 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074933.961323:0:10283:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549279800, rc 0) 00000100:00100000:3.0:1769074933.961438:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68531:x1855007617071232:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 774us (1016us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074933.961447:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13683 00000100:00100000:1.0:1769074933.968667:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074933.968742:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617071360 00000100:00100000:3.0:1769074933.968753:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617071360 00000100:00100000:3.0:1769074933.968836:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13684 00000100:00100000:3.0:1769074933.968847:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68531:x1855007617071360:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:3.0:1769074933.968933:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:3.0:1769074933.969156:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:3.0:1769074933.969171:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969241:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074933.969250:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969257:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969331:0:10283:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10283 initiator: MDT0 00010000:00010000:0.0:1769074933.969339:0:10283:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969346:0:10283:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00000004:00200000:0.0:1769074933.969384:0:10283:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:0.0:1769074933.969389:0:10283:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074933.969400:0:10283:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969413:0:10283:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969419:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969425:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969443:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969450:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.969454:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969460:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969471:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea900) 00010000:00010000:0.0:1769074933.969474:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074933.969477:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969483:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279c00/0x7fe2323adfd560bf lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969501:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969506:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969511:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969516:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.969519:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969523:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969529:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b67a317200) 00010000:00010000:0.0:1769074933.969531:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074933.969533:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969537:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134000/0x7fe2323adfd560c6 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969547:0:10283:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969551:0:10283:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 2/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969556:0:10283:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969560:0:10283:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074933.969562:0:10283:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969567:0:10283:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 4/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969573:0:10283:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:0.0:1769074933.969575:0:10283:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074933.969577:0:10283:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 2/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00010000:00010000:0.0:1769074933.969582:0:10283:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134a00/0x7fe2323adfd560cd lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10283 initiator: MDT1 00000100:00100000:0.0:1769074933.969660:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54441ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68531:x1855007617071360:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 813us (1001us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074933.969667:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13684 00010000:00010000:0.0:1769074934.173443:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074935.201485:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074935.709943:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074935.709976:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625284224:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710046:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074935.710095:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284224 00000100:00100000:0.0:1769074935.710102:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284224 00000100:00100000:0.0:1769074935.710142:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:0.0:1769074935.710151:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fb8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625284224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710213:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fb8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625284224:12345-0@lo:400:kworker.0 Request processed in 61us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074935.710233:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:0.0:1769074935.710279:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625284224:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710767:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625284480:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.710773:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625284352:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710799:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.710799:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074935.710835:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625284992:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.710835:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625284864:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710856:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074935.710877:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625285504:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.710877:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625284608:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710897:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.710899:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.710914:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625285376:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.710927:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284480 00000100:00100000:0.0:1769074935.710931:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284480 00000100:00100000:0.0:1769074935.710959:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284992 00000100:00100000:0.0:1769074935.710962:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284992 00000100:00100000:0.0:1769074935.710985:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284864 00000100:00100000:0.0:1769074935.710988:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284864 00000100:00100000:0.0:1769074935.711001:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2575 00000100:00100000:0.0:1769074935.711007:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625284480:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.711061:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625284480:12345-0@lo:400:kworker.0 Request processed in 53us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074935.711068:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2575 00000100:00100000:0.0:1769074935.711083:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2576 00000100:00100000:0.0:1769074935.711089:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625284992:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.711115:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625284992:12345-0@lo:400:kworker.0 Request processed in 27us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074935.711121:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2576 00000100:00100000:0.0:1769074935.711132:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2577 00000100:00100000:0.0:1769074935.711137:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625284864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.711148:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074935.711163:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625284864:12345-0@lo:400:kworker.0 Request processed in 27us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074935.711168:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2577 00000100:00100000:1.0:1769074935.711181:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625284864:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.711191:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285504 00000100:00100000:0.0:1769074935.711194:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285504 00000100:00080000:1.0:1769074935.711208:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074935.711213:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074935.711214:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074935.711228:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.711246:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625285120:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.711268:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.711283:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625285632:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.711302:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.711317:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1769074935.711321:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625284736:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.711341:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.711361:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625285248:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.711379:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.711394:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625285760:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.711413:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074935.711432:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284608 00000100:00100000:1.0:1769074935.711435:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284608 00000100:00100000:0.0:1769074935.711655:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1633 00000100:00100000:0.0:1769074935.711665:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625285504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.711699:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625285504:12345-0@lo:400:kworker.0 Request processed in 37us (801us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074935.711912:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285632 00000100:00100000:1.0:1769074935.711918:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285632 00000100:00100000:2.0:1769074935.711969:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284352 00000100:00100000:2.0:1769074935.711975:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284352 00000100:00100000:2.0:1769074935.712004:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285376 00000100:00100000:2.0:1769074935.712007:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285376 00000100:00100000:2.0:1769074935.712038:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2792 00000100:00100000:2.0:1769074935.712044:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625284352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074935.712088:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625284352:12345-0@lo:400:kworker.0 Request processed in 45us (1288us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712094:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2792 00000100:00100000:2.0:1769074935.712109:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2793 00000100:00100000:2.0:1769074935.712116:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625285376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712134:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625284736 00000100:00100000:1.0:1769074935.712138:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625284736 00000100:00100000:2.0:1769074935.712143:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625285376:12345-0@lo:400:kworker.0 Request processed in 28us (995us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712149:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2793 00000100:00100000:1.0:1769074935.712158:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1634 00000100:00100000:1.0:1769074935.712164:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625284608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074935.712165:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285120 00000100:00100000:2.0:1769074935.712168:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285120 00000100:00100000:2.0:1769074935.712197:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285248 00000100:00100000:1.0:1769074935.712199:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625284608:12345-0@lo:400:kworker.0 Request processed in 35us (971us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712200:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285248 00000100:00100000:1.0:1769074935.712205:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1634 00000100:00100000:2.0:1769074935.712215:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285760 00000100:00100000:2.0:1769074935.712217:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285760 00000100:00100000:1.0:1769074935.712218:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1635 00000100:00100000:1.0:1769074935.712223:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625285632:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074935.712239:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2578 00000100:00100000:2.0:1769074935.712245:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625285120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712248:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625285632:12345-0@lo:400:kworker.0 Request processed in 25us (945us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074935.712253:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1635 00000100:00100000:1.0:1769074935.712264:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1636 00000100:00100000:1.0:1769074935.712270:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625284736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074935.712272:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625285120:12345-0@lo:400:kworker.0 Request processed in 27us (1004us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712277:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2578 00000100:00100000:2.0:1769074935.712288:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2579 00000100:00100000:1.0:1769074935.712291:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+9:3649:x1855007625284736:12345-0@lo:400:kworker.0 Request processed in 23us (950us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712293:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625285248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712297:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1636 00000100:00100000:2.0:1769074935.712316:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625285248:12345-0@lo:400:kworker.0 Request processed in 22us (935us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712321:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2579 00000100:00100000:1.0:1769074935.712327:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2580 00000100:00100000:1.0:1769074935.712334:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625285760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712360:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625285760:12345-0@lo:400:kworker.0 Request processed in 28us (947us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074935.712363:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625284352:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712366:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2580 00000100:00100000:2.0:1769074935.712372:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625285376:0@lo:400:kworker.0 00000100:00080000:1.0:1769074935.712380:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074935.712383:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074935.712385:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074935.712387:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074935.712403:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625284736:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712412:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625285248:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712422:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625285760:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712430:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625284608:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712439:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625285120:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712447:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625285632:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712507:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625284480:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712517:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625284992:0@lo:400:kworker.0 00000100:00100000:1.0:1769074935.712526:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625285504:0@lo:400:kworker.0 00000100:00100000:0.0:1769074935.714833:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1633 00010000:00010000:1.0:1769074936.224127:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074936.772688:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074936.772867:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074936.772885:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074936.772916:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074936.772944:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072000 00000100:00100000:0.0:1769074936.772947:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074936.772962:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072000 00000100:00100000:3.0:1769074936.773022:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617071872 00000100:00100000:3.0:1769074936.773026:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617071872 00000100:00100000:3.0:1769074936.773054:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1637 00000100:00100000:3.0:1769074936.773062:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617072000:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074936.773194:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617072000:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 131us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074936.773204:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1637 00000100:00100000:1.0:1769074936.773219:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617071744 00000100:00100000:3.0:1769074936.773221:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1638 00000100:00100000:3.0:1769074936.773228:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617071872:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074936.773234:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617071744 00000100:00100000:3.0:1769074936.773256:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+12:2406:x1855007617071872:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 29us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074936.773262:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1638 00000100:00100000:1.0:1769074936.773274:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617071616 00000100:00100000:1.0:1769074936.773277:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617071616 00000100:00100000:1.0:1769074936.773300:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2581 00000100:00100000:3.0:1769074936.773305:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2582 00000100:00100000:1.0:1769074936.773307:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617071744:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074936.773311:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007617071616:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074936.773338:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:2406:x1855007617071616:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 27us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074936.773342:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2406:x1855007617071744:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 37us (476us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074936.773344:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2582 00000100:00100000:1.0:1769074936.773348:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2581 00000100:00080000:1.0:1769074936.773371:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074936.773375:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074936.773377:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074936.773379:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074936.781508:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617071488 00000100:00100000:2.0:1769074936.781523:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617071488 00000100:00100000:3.0:1769074936.781788:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 407 00000100:00100000:3.0:1769074936.781823:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b644fb9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617071488:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074936.781897:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b644fb9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617071488:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 99us (9219us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074936.781905:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 407 00010000:00010000:0.0:1769074937.251373:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074937.757042:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074937.757069:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625285888:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074937.757138:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074937.757260:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625285888 00000100:00100000:3.0:1769074937.757269:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625285888 00000100:00100000:3.0:1769074937.757325:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2794 00000100:00100000:3.0:1769074937.757334:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625285888:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074937.757369:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:3.0:1769074937.757427:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625285888:12345-0@lo:41:osp-pre-1-0.0 Request processed in 94us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074937.757434:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2794 00000100:00100000:0.0:1769074937.757515:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625285888:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074937.767222:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074937.767246:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625286016:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074937.767315:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074937.767364:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286016 00000100:00100000:2.0:1769074937.767371:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286016 00000100:00100000:2.0:1769074937.767418:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2795 00000100:00100000:2.0:1769074937.767426:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625286016:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1769074937.767459:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:2.0:1769074937.767503:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625286016:12345-0@lo:41:osp-pre-0-1.0 Request processed in 77us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074937.767510:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2795 00000100:00100000:2.0:1769074937.767567:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625286016:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1769074938.269187:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1769074938.269210:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625286144:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074938.269282:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074938.269643:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286144 00000100:00100000:3.0:1769074938.269652:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286144 00000100:00100000:3.0:1769074938.269718:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1296 00000100:00100000:3.0:1769074938.269728:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625286144:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074938.269756:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3913945088 avail 3682422784 00000100:00100000:3.0:1769074938.269836:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625286144:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (524us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074938.269844:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1296 00000100:00100000:3.0:1769074938.269895:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625286144:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1769074938.270227:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1769074938.270587:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074938.270648:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074938.270662:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625286272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769074938.270705:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074938.271772:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625286400:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074938.271838:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074938.271931:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286272 00000100:00100000:0.0:1769074938.271938:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286272 00000100:00100000:0.0:1769074938.271982:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286400 00000100:00100000:0.0:1769074938.271985:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286400 00000100:00100000:0.0:1769074938.272008:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1297 00000100:00100000:0.0:1769074938.272014:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625286272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769074938.272041:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 60273984 pending 1073152 free 3914907648 avail 3683385344 00000100:00100000:0.0:1769074938.272085:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625286272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (1379us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074938.272093:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1297 00000100:00100000:0.0:1769074938.272110:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1298 00000100:00100000:0.0:1769074938.272116:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625286400:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074938.272125:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3913945088 avail 3682422784 00000100:00100000:0.0:1769074938.272155:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625286400:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074938.272161:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1298 00000100:00080000:0.0:1769074938.272182:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074938.272186:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074938.272217:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625286400:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074938.272250:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625286272:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1769074938.274125:0:32377:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 8589944144 is committed 00000100:00100000:0.0:1769074938.782663:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074938.782687:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625286528:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1769074938.782756:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074938.783566:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286528 00000100:00100000:2.0:1769074938.783577:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286528 00000100:00100000:2.0:1769074938.783642:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1299 00000100:00100000:2.0:1769074938.783650:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+9:3651:x1855007625286528:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1769074938.783676:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 60273984 pending 1073152 free 3914907648 avail 3683385344 00000100:00100000:2.0:1769074938.783737:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+9:3651:x1855007625286528:12345-0@lo:13:osp-pre-1-1.0 Request processed in 87us (980us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074938.783743:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1299 00000100:00080000:2.0:1769074938.783765:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074938.783824:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625286528:0@lo:13:osp-pre-1-1.0 00000001:00080000:0.0:1769074939.315422:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841321 is committed 00000001:00080000:0.0:1769074939.315447:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841319 is committed 00000001:00080000:0.0:1769074939.315488:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968390 is committed 00010000:00010000:2.0:1769074939.321547:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1769074939.327668:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874652 is committed 00000001:00080000:3.0:1769074939.327686:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874650 is committed 00000100:00100000:1.0:1769074939.754318:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074939.757629:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072128 00000100:00100000:3.0:1769074939.757647:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072128 00000100:00100000:3.0:1769074939.757731:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13685 00000100:00100000:3.0:1769074939.757741:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68956:x1855007617072128:12345-192.168.204.60@tcp:101:rm.0 00010000:00010000:3.0:1769074939.757761:0:6513:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074939.757785:0:6513:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279200/0x7fe2323adfd560d4 lrc: 2/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca93 expref: -99 pid: 6513 timeout: 0 lvb_type: 0 00000100:00080000:1.0:1769074939.757824:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1769074939.757868:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 3/1,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074939.757882:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.757893:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 3/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.757906:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074939.757913:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.757921:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.757996:0:6513:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 3/1,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6513 initiator: MDT0 00010000:00010000:3.0:1769074939.758005:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758011:0:6513:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00000004:00200000:3.0:1769074939.758063:0:6513:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000404:0x4d:0x0]: returning size 70254592 00000004:00200000:3.0:1769074939.758068:0:6513:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1769074939.758089:0:6513:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758102:0:6513:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 3/1,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758111:0:6513:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758117:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 4/1,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758127:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 3/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758134:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758140:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074939.758143:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 4/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758151:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 5/0,0 mode: CR/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758164:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b6543ea700) 00010000:00010000:3.0:1769074939.758166:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074939.758172:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 3/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758177:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279600/0x7fe2323adfd560db lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758188:0:6513:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758193:0:6513:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 2/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758199:0:6513:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758204:0:6513:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074939.758206:0:6513:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758216:0:6513:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 4/0,0 mode: PR/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758222:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b6543ea000) 00010000:00010000:3.0:1769074939.758225:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074939.758227:0:6513:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 2/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758232:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279800/0x7fe2323adfd560e2 lrc: 0/0,0 mode: --/PR res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6513 initiator: MDT1 00010000:00010000:3.0:1769074939.758262:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549279200/0x7fe2323adfd560d4 lrc: 0/0,0 mode: --/CR res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca93 expref: 11 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074939.758275:0:6513:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 3/0,0 mode: PR/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca93 expref: 10 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074939.758281:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:3.0:1769074939.758283:0:6513:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769074939.758285:0:6513:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549278200, rc 0) 00000100:00100000:3.0:1769074939.758360:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54ce9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68956:x1855007617072128:12345-192.168.204.60@tcp:101:rm.0 Request processed in 620us (4049us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074939.758367:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13685 00000100:00100000:0.0:1769074939.769469:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074939.769605:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072384 00000100:00100000:1.0:1769074939.769614:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072384 00000100:00100000:1.0:1769074939.769672:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13686 00000100:00100000:1.0:1769074939.769682:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:68956:x1855007617072384:12345-192.168.204.60@tcp:36:rm.0 00010000:00010000:1.0:1769074939.769735:0:6514:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1769074939.769760:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243600/0x7fe2323adfd55fd1 lrc: 0/0,0 mode: --/CR res: [0x240000404:0x4d:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca62 expref: 10 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074939.769780:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549278200/0x7fe2323adfd560e9 lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca93 expref: 9 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074939.769794:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135000/0x7fe2323adfd55e1f lrc: 0/0,0 mode: --/PR res: [0x240000404:0x4d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c9cf expref: 8 pid: 6513 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074939.769823:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b6543ea600) 00010000:00010000:1.0:1769074939.769836:0:6514:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769074939.769862:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 3/0,1 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074939.769875:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 3/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.769884:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 3/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.769906:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074939.769914:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.769921:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770007:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 3/0,1 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074939.770027:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770034:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00000040:00080000:1.0:1769074939.770331:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].72, 64 off13952 00000004:00080000:1.0:1769074939.770342:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1927612534197ns, next at 1928612531825ns, rc = 0 00000040:00080000:1.0:1769074939.770363:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].77, 64 off14296 00000004:00080000:1.0:1769074939.770369:0:6514:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1927612561924ns, next at 1928612560760ns, rc = 0 00000040:00080000:1.0:1769074939.770379:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].78, 64 off14360 00000040:00080000:1.0:1769074939.770389:0:6514:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].73, 64 off14016 00010000:00010000:1.0:1769074939.770441:0:6514:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 3/0,1 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6514 initiator: MDT0 00010000:00010000:1.0:1769074939.770451:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770458:0:6514:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 3/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770464:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 4/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770470:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 4/0,1 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770475:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770498:0:6514:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1769074939.770523:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 3/0,1 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770530:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 2/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770536:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770542:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074939.770546:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 3/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770553:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 4/0,0 mode: EX/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770559:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1769074939.770562:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b681697400) 00010000:00010000:1.0:1769074939.770564:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1769074939.770565:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 2/0,0 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770568:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 2/0,0 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770572:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242a00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:1.0:1769074939.770573:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242c00/0x7fe2323adfd560fe lrc: 0/0,0 mode: --/EX res: [0x240000404:0x4d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770575:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 2/0,0 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770581:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 3/0,0 mode: PW/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770588:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000404:0x4d:0x0].0x0 (ffff89b681697400) 00010000:00010000:2.0:1769074939.770590:0:6502:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1769074939.770593:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 1/0,0 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770598:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242a00/0x7fe2323adfd56105 lrc: 0/0,0 mode: --/PW res: [0x240000404:0x4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6514 initiator: MDT1 00010000:00010000:2.0:1769074939.770610:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074939.770612:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074939.770614:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074939.770686:0:6514:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 4/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770692:0:6514:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 4/0,1 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770697:0:6514:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 3/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770702:0:6514:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 4/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770706:0:6514:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243200 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:1.0:1769074939.770709:0:6514:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 4/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770715:0:6514:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 5/0,0 mode: CW/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770722:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0x0 (ffff89b681697700) 00010000:00010000:1.0:1769074939.770725:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769074939.770728:0:6514:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 3/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770734:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549243200/0x7fe2323adfd560f0 lrc: 0/0,0 mode: --/CW res: [0x240000405:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00000004:00010000:1.0:1769074939.770744:0:6514:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000054ed942 reply state 000000000b2424a9 transno 4294968391 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,1 mode: PW/PW res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770752:0:6514:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:1.0:1769074939.770758:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b681697300) 00010000:00010000:1.0:1769074939.770760:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1769074939.770762:0:6514:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1769074939.770853:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:68956:x1855007617072384:12345-192.168.204.60@tcp:36:rm.0 Request processed in 1170us (1393us total) trans 4294968391 rc 0/0 00000100:00100000:1.0:1769074939.770862:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13686 00010000:00010000:2.0:1769074940.382573:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074940.829044:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625287040:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.829111:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074940.829158:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625287552:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.829182:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074940.829197:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625288064:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.829216:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074940.829239:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:1.0:1769074940.829244:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625286912:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.829263:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074940.829277:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625287424:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.829295:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074940.829322:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625287936:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.829342:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074940.829369:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287040 00000100:00100000:1.0:1769074940.829375:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287040 00000100:00100000:1.0:1769074940.829419:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288064 00000100:00100000:1.0:1769074940.829423:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288064 00000100:00100000:2.0:1769074940.829925:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625286656:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.829964:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.829989:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625287168:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.830009:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.830025:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625287680:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.830044:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.830061:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67a0f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625288192:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.830079:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.830096:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1769074940.830101:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625286784:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.830121:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.830135:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625287296:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.830153:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.830162:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625287808:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.830178:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074940.830206:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287552 00000100:00100000:2.0:1769074940.830211:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287552 00000100:00100000:2.0:1769074940.830235:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286912 00000100:00100000:2.0:1769074940.830238:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286912 00000100:00100000:2.0:1769074940.830248:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287424 00000100:00100000:2.0:1769074940.830250:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287424 00000100:00100000:2.0:1769074940.830261:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287680 00000100:00100000:2.0:1769074940.830263:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287680 00000100:00100000:2.0:1769074940.830273:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288192 00000100:00100000:2.0:1769074940.830275:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288192 00000100:00100000:2.0:1769074940.830285:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287296 00000100:00100000:2.0:1769074940.830287:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287296 00000100:00100000:2.0:1769074940.830302:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2583 00000100:00100000:1.0:1769074940.830963:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287936 00000100:00100000:1.0:1769074940.830970:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287936 00000100:00100000:1.0:1769074940.831009:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287168 00000100:00100000:1.0:1769074940.831011:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287168 00000100:00100000:1.0:1769074940.831029:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1639 00000100:00100000:1.0:1769074940.831037:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625287040:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831102:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625287040:12345-0@lo:400:kworker.0 Request processed in 65us (1991us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831110:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1639 00000100:00100000:1.0:1769074940.831129:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1640 00000100:00100000:1.0:1769074940.831137:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625288064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831167:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625288064:12345-0@lo:400:kworker.0 Request processed in 31us (1950us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831172:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1640 00000100:00100000:1.0:1769074940.831183:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1641 00000100:00100000:1.0:1769074940.831188:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625287936:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831212:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549203800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625287936:12345-0@lo:400:kworker.0 Request processed in 24us (1869us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831217:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1641 00000100:00100000:1.0:1769074940.831228:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1642 00000100:00100000:1.0:1769074940.831233:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625287168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831264:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625287168:12345-0@lo:400:kworker.0 Request processed in 31us (1254us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831270:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1642 00000100:00100000:2.0:1769074940.831352:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625287552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.831384:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a0f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1855007625287552:12345-0@lo:400:kworker.0 Request processed in 36us (2202us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831747:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2584 00000100:00100000:1.0:1769074940.831757:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549200a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625286912:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831793:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549200a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1855007625286912:12345-0@lo:400:kworker.0 Request processed in 38us (2528us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831798:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2584 00000100:00100000:1.0:1769074940.831833:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2585 00000100:00100000:1.0:1769074940.831852:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625287424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831882:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549201f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1855007625287424:12345-0@lo:400:kworker.0 Request processed in 31us (2586us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831887:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2585 00000100:00100000:1.0:1769074940.831899:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2586 00000100:00100000:1.0:1769074940.831905:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625287680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831932:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625287680:12345-0@lo:400:kworker.0 Request processed in 28us (1887us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831939:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2586 00000100:00100000:1.0:1769074940.831951:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2587 00000100:00100000:1.0:1769074940.831956:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625288192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.831979:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625288192:12345-0@lo:400:kworker.0 Request processed in 23us (1899us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.831984:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2587 00000100:00100000:1.0:1769074940.831995:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2588 00000100:00100000:1.0:1769074940.832001:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625287296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832028:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625287296:12345-0@lo:400:kworker.0 Request processed in 27us (1874us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.832034:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2588 00000100:00080000:1.0:1769074940.832052:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074940.832056:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074940.832059:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074940.832061:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074940.832110:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3652:1855007625286912:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832122:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3652:1855007625287424:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832133:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625287936:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832144:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 21474841321 gen 2 00000004:00080000:1.0:1769074940.832149:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c68a80, transno 21474841320 00000004:00080000:1.0:1769074940.832159:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54441fb80, transno 21474841321 00000100:00100000:1.0:1769074940.832164:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625287040:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832174:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3652:1855007625287552:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832185:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625288064:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.832390:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286784 00000100:00100000:1.0:1769074940.832396:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286784 00000100:00100000:1.0:1769074940.832424:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625287808 00000100:00100000:1.0:1769074940.832427:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625287808 00000100:00100000:1.0:1769074940.833872:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2796 00000100:00100000:1.0:1769074940.833888:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625286784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.833963:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625286784:12345-0@lo:400:kworker.0 Request processed in 80us (3840us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.833970:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2796 00000100:00100000:1.0:1769074940.833988:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2797 00000100:00100000:1.0:1769074940.833995:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625287808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.834026:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625287808:12345-0@lo:400:kworker.0 Request processed in 31us (3845us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.834031:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2797 00000100:00100000:1.0:1769074940.834129:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625286784:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.834142:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625287296:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.834152:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625287808:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.834162:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 17179874652 gen 2 00000004:00080000:1.0:1769074940.834167:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54c846d80, transno 17179874648 00000004:00080000:1.0:1769074940.834177:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b54c845f80, transno 17179874649 00000004:00080000:1.0:1769074940.834180:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ece00, transno 17179874651 00000004:00080000:1.0:1769074940.834182:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491edf80, transno 17179874652 00000100:00100000:1.0:1769074940.834187:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d0e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625287168:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.834198:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54441ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625287680:0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.834209:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67a0f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625288192:0@lo:400:kworker.0 00000100:00100000:2.0:1769074940.837186:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2583 00000004:00080000:1.0:1769074940.837555:0:9490:0:(osp_sync.c:1179:osp_sync_process_committed()) lustre-OST0001-osc-MDT0001: 2 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:1.0:1769074940.837598:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625286656 00000100:00100000:1.0:1769074940.837606:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625286656 00000100:00100000:1.0:1769074940.837650:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1769074940.837659:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625286656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074940.837747:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625286656:12345-0@lo:400:kworker.0 Request processed in 89us (7781us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074940.837754:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1769074940.838388:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54ce9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625286656:0@lo:400:kworker.0 00000001:02000400:2.0:1769074941.239612:0:39345:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 204e: FLR write/stale/resync sel test with repeated comp ========================================================== 04:42:19 (1769074939) 00010000:00010000:2.0:1769074941.404956:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074941.897462:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074941.897563:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074941.897592:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074941.897649:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074941.897671:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074941.897687:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072512 00000100:00100000:1.0:1769074941.897696:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072512 00000100:00100000:1.0:1769074941.897729:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 409 00000100:00100000:1.0:1769074941.897735:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617072512:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074941.897797:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2408:x1855007617072512:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 61us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074941.897826:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 409 00000100:00100000:3.0:1769074941.897894:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072640 00000100:00100000:3.0:1769074941.897899:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072640 00000100:00100000:3.0:1769074941.897956:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072768 00000100:00100000:3.0:1769074941.897959:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072768 00000100:00100000:3.0:1769074941.897981:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2589 00000100:00100000:3.0:1769074941.897988:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c846d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007617072640:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074941.898024:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c846d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2409:x1855007617072640:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 37us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074941.898030:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2589 00000100:00100000:3.0:1769074941.898042:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2590 00000100:00100000:3.0:1769074941.898048:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617072768:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074941.898077:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54c845f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617072768:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 28us (484us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074941.898083:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2590 00000100:00100000:1.0:1769074941.898581:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617072896 00000100:00100000:1.0:1769074941.898587:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617072896 00000100:00100000:1.0:1769074941.898621:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073024 00000100:00100000:1.0:1769074941.898624:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073024 00000100:00100000:1.0:1769074941.898647:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1643 00000100:00100000:1.0:1769074941.898656:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617072896:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074941.898695:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617072896:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 41us (1043us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074941.898702:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1643 00000100:00100000:1.0:1769074941.898717:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1644 00000100:00100000:1.0:1769074941.898724:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b5e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617073024:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074941.898757:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b5e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2408:x1855007617073024:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 34us (1084us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074941.898767:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1644 00000100:00080000:1.0:1769074941.899601:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074941.899607:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074941.899610:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074941.899612:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1769074942.431098:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074942.877998:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074942.878022:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625288320:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074942.878090:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074942.878262:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288320 00000100:00100000:0.0:1769074942.878558:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288320 00000100:00100000:0.0:1769074942.878619:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2798 00000100:00100000:0.0:1769074942.878628:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625288320:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1769074942.878659:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:0.0:1769074942.878951:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625288320:12345-0@lo:41:osp-pre-1-0.0 Request processed in 324us (861us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074942.878959:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2798 00000100:00100000:0.0:1769074942.879006:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625288320:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074942.881305:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769074942.881329:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625288448:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074942.881404:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074942.881732:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288448 00000100:00100000:1.0:1769074942.881745:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288448 00000100:00100000:1.0:1769074942.882097:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2799 00000100:00100000:1.0:1769074942.882107:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625288448:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074942.882144:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:1.0:1769074942.882207:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625288448:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (803us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074942.882215:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2799 00000100:00100000:3.0:1769074942.882311:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625288448:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074943.395917:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074943.395938:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625288704:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074943.396003:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074943.396082:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288704 00000100:00100000:2.0:1769074943.396090:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288704 00000100:00100000:2.0:1769074943.396146:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1300 00000100:00100000:2.0:1769074943.396154:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625288704:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074943.396184:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3913945088 avail 3682422784 00000100:00100000:2.0:1769074943.396226:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625288704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074943.396233:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1300 00000100:00100000:2.0:1769074943.396286:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625288704:0@lo:13:osp-pre-0-0.0 00000004:00080000:1.0:1769074943.398313:0:9036:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1931240502901ns, next at 1932240500571ns, rc = 0 00000100:00080000:1.0:1769074943.398339:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074943.398375:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074943.398395:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625288832:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074943.398465:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074943.398521:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1769074943.398531:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625288576:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074943.398556:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074943.398587:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288832 00000100:00100000:1.0:1769074943.398594:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288832 00000100:00100000:1.0:1769074943.398646:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288576 00000100:00100000:1.0:1769074943.398649:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288576 00000100:00100000:1.0:1769074943.398671:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1301 00000100:00100000:1.0:1769074943.398932:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625288832:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074943.398959:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3913945088 avail 3682422784 00000100:00100000:1.0:1769074943.399004:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625288832:12345-0@lo:13:osp-pre-0-1.0 Request processed in 325us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074943.399011:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1301 00000100:00100000:1.0:1769074943.399033:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1302 00000100:00100000:1.0:1769074943.399039:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625288576:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074943.399055:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3914907648 avail 3683385344 00000100:00100000:1.0:1769074943.399085:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625288576:12345-0@lo:13:osp-pre-1-0.0 Request processed in 47us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074943.399092:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1302 00000100:00080000:1.0:1769074943.399108:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074943.399111:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074943.399147:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625288832:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074943.399186:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625288576:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1769074943.453196:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1769074943.461527:0:39441:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1931303716625ns, next at 1881035490390ns, rc = 0 00080000:00080000:0.0:1769074943.461547:0:39441:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:0.0:1769074943.462090:0:39441:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 1931304281935ns, next at 1932240500571ns, rc = 0 00080000:00080000:0.0:1769074943.462184:0:39441:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:1.0:1769074943.469169:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 1975 committed 00000004:00080000:2.0:1769074943.470709:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 73 committed 00000004:00080000:2.0:1769074943.470783:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 72 committed 00000100:00100000:0.0:1769074943.472268:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625289088:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1769074943.472342:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074943.472395:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074943.472402:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625288960:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1769074943.472424:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074943.472541:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625289088 00000100:00100000:1.0:1769074943.472549:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625289088 00000100:00100000:1.0:1769074943.472601:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625288960 00000100:00100000:1.0:1769074943.472604:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625288960 00000100:00100000:1.0:1769074943.472626:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14568 00000100:00100000:1.0:1769074943.472634:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+6:3652:x1855007625289088:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1769074943.472662:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:55 count 1 00010000:00010000:1.0:1769074943.472690:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,1 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.472710:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.472719:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074943.472721:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074943.472725:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 4/0,1 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.472754:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.472768:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 9 pid: 8398 timeout: 2031 lvb_type: 0 00000100:00100000:1.0:1769074943.472781:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625289216:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074943.472863:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5437ba3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769074943.472878:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14569 00000100:00100000:1.0:1769074943.472885:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3652:x1855007625288960:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1769074943.472900:0:28519:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:54 count 1 00010000:00010000:1.0:1769074943.472912:0:28519:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.472922:0:28519:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.472928:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074943.472929:0:28519:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769074943.472931:0:28519:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.473002:0:28519:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 9 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1769074943.473011:0:28519:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 9 pid: 8398 timeout: 2031 lvb_type: 0 00000100:00100000:1.0:1769074943.473019:0:28519:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625289344:192.168.204.60@tcp:104: 00000100:00100000:1.0:1769074943.473045:0:28519:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b5437baa80 going to sleep for 11 seconds 00000100:00100000:0.0:1769074943.476498:0:28519:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:28519:1855007625289344:192.168.204.60@tcp:104: 00010000:00010000:0.0:1769074943.476529:0:28519:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074943.476661:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5460ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:27843:1855007625289216:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074943.476674:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,1 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00000001:00080000:2.0:1769074943.476786:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968391 is committed 00000004:00080000:2.0:1769074943.476803:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 76 committed 00000004:00080000:2.0:1769074943.476831:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 71 committed 00000100:00100000:3.0:1769074943.477034:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074943.477051:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625289472:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1769074943.477099:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074943.477150:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625289472 00000100:00100000:3.0:1769074943.477157:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625289472 00000100:00100000:3.0:1769074943.477208:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14570 00000100:00100000:3.0:1769074943.477215:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625289472:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1769074943.477240:0:36516:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:56 count 1 00010000:00010000:3.0:1769074943.477276:0:36516:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.477293:0:36516:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x40000000020000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 8 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.477301:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074943.477303:0:36516:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074943.477306:0:36516:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.477333:0:36516:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x50000000030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 8 pid: 8398 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.477346:0:36516:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x70000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 8 pid: 8398 timeout: 2031 lvb_type: 0 00000100:00100000:3.0:1769074943.477357:0:36516:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625289600:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074943.477413:0:36516:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442c600 going to sleep for 11 seconds 00000100:00080000:3.0:1769074943.477425:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074943.478103:0:3635:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074943.478113:0:3635:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074943.478129:0:3635:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074943.478136:0:3635:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549242000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074943.478140:0:3635:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074943.478148:0:3635:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074943.478164:0:3635:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xa:0x0].0xc2cf826c (ffff89b681697300) 00010000:00010000:0.0:1769074943.478169:0:3635:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074943.478173:0:3635:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 2/0,0 mode: --/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00010000:00010000:0.0:1769074943.478179:0:3635:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549242000/0x7fe2323adfd560f7 lrc: 0/0,0 mode: --/TXN res: [0x240000405:0xa:0x0].0xc2cf826c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6514 initiator: MDT1 00000100:00100000:1.0:1769074943.480640:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074943.480688:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074943.480761:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073280 00000100:00100000:0.0:1769074943.480770:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073280 00010000:00010000:0.0:1769074943.480866:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 10 pid: 8398 timeout: 2031 lvb_type: 0 00000100:00100000:0.0:1769074943.480893:0:8424:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073152 00000100:00100000:0.0:1769074943.480896:0:8424:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073152 00010000:00010000:0.0:1769074943.480908:0:8424:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 11 pid: 8398 timeout: 2031 lvb_type: 0 00000100:00100000:0.0:1769074943.480929:0:8424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7240 00000100:00100000:0.0:1769074943.480938:0:8424:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+11:2408:x1855007617073280:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074943.480952:0:8424:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074943.480957:0:8424:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 11 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:0.0:1769074943.480968:0:8424:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 11 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:0.0:1769074943.480991:0:8424:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd55fbc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->67108863) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca54 expref: 11 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:0.0:1769074943.481013:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074943.481016:0:8424:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074943.481020:0:8424:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.481029:0:8424:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.481057:0:8424:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074943.481104:0:8424:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2408:x1855007617073280:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 167us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074943.481111:0:8424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7240 00000100:00100000:2.0:1769074943.481211:0:33102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7241 00000100:00100000:2.0:1769074943.481223:0:33102:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b64effdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+10:2409:x1855007617073152:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074943.481230:0:33102:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074943.481235:0:33102:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 10 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:2.0:1769074943.481244:0:33102:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 10 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:2.0:1769074943.481256:0:33102:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd55fae lrc: 0/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 10485760->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca4d expref: 10 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:2.0:1769074943.481276:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074943.481278:0:33102:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074943.481281:0:33102:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.481288:0:33102:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.481304:0:33102:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074943.481324:0:33102:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b64effdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617073152:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 105us (636us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074943.481331:0:33102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7241 00010000:00010000:0.0:1769074943.481574:0:28519:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.481584:0:28519:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.481595:0:28519:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.481603:0:28519:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.481627:0:28519:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1769074943.481920:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625288960:12345-0@lo:6:osp-syn-0-1.0 Request processed in 9034us (9493us total) trans 21474841322 rc 0/0 00000100:00100000:0.0:1769074943.481931:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14569 00000004:00080000:0.0:1769074943.481979:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546c69880/2, rc 0, transno 21474841322 00000100:00100000:0.0:1769074943.481990:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625288960:0@lo:6:osp-syn-0-1.0 00000100:00080000:1.0:1769074943.483370:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1769074943.483978:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.483992:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.484007:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.484015:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.484044:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1769074943.485140:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625289088:12345-0@lo:6:osp-syn-0-1.0 Request processed in 12501us (12792us total) trans 21474841323 rc 0/0 00000100:00100000:3.0:1769074943.485151:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14568 00010000:00010000:0.0:1769074943.485355:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1769074943.485365:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485373:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243e00 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074943.485377:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485386:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485405:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485413:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243e00/0x7fe2323adfd56113 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 28519 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485429:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1769074943.485432:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485437:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549243800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074943.485440:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485458:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485468:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 1/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485475:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: ffff89b549243800/0x7fe2323adfd5610c lrc: 0/0,0 mode: --/PW res: [0x280000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.485488:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074943.485490:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074943.485492:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074943.485560:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1769074943.486527:0:3652:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b546c69c00/2, rc 0, transno 21474841323 00000100:00100000:1.0:1769074943.486550:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625289088:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1769074943.486956:0:39441:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1931329148268ns, next at 1881035908970ns, rc = 0 00080000:00080000:0.0:1769074943.486970:0:39441:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:0.0:1769074943.487050:0:39441:0:(osp_sync.c:1775:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 1931329242620ns, next at 1928612560760ns, rc = 0 00080000:00080000:0.0:1769074943.487056:0:39441:0:(osd_handler.c:2876:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:2.0:1769074943.491500:0:7431:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 77 committed 00000100:00100000:1.0:1769074943.492140:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074943.492163:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625289728:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1769074943.492238:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074943.492407:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625289728 00000100:00100000:3.0:1769074943.492415:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625289728 00000100:00100000:3.0:1769074943.492478:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14571 00000100:00100000:3.0:1769074943.492487:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+6:3649:x1855007625289728:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1769074943.492520:0:27843:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:55 count 1 00010000:00010000:3.0:1769074943.492550:0:27843:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.492572:0:27843:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x40000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.492579:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074943.492581:0:27843:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:3.0:1769074943.492585:0:27843:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.492612:0:27843:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000000010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 8 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.492625:0:27843:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x70000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 8 pid: 36516 timeout: 2031 lvb_type: 0 00000100:00100000:3.0:1769074943.492635:0:27843:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625289856:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074943.492706:0:27843:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54442c6c0 going to sleep for 11 seconds 00000004:00080000:1.0:1769074943.494202:0:32377:0:(osp_sync.c:1740:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1979 committed 00000100:00100000:1.0:1769074943.496032:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074943.496167:0:36516:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:36516:1855007625289600:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074943.496202:0:36516:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1769074943.496293:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073408 00000100:00100000:2.0:1769074943.496301:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073408 00010000:00010000:2.0:1769074943.496353:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 9 pid: 8398 timeout: 2031 lvb_type: 0 00000100:00100000:2.0:1769074943.496390:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7242 00000100:00100000:2.0:1769074943.496399:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+9:2409:x1855007617073408:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074943.496414:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074943.496420:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x60000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 9 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:2.0:1769074943.496431:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x50000400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 9 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:2.0:1769074943.496466:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b666b88000/0x7fe2323adfd55fc3 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 67108864->134217727) gid 0 flags: 0x44801400030020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca5b expref: 9 pid: 8398 timeout: 2031 lvb_type: 0 00010000:00010000:2.0:1769074943.496481:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074943.496483:0:6500:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769074943.496488:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.496498:0:6500:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.496531:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074943.496579:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617073408:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 181us (556us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074943.496587:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7242 00010000:00010000:0.0:1769074943.496975:0:36516:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.496984:0:36516:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.496996:0:36516:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.497003:0:36516:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.497027:0:36516:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00080000:1.0:1769074943.497925:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074943.499158:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625289472:12345-0@lo:6:osp-syn-1-1.0 Request processed in 21936us (22051us total) trans 17179874653 rc 0/0 00000100:00100000:0.0:1769074943.499171:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14570 00010000:00010000:2.0:1769074943.499279:0:8413:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1769074943.499289:0:8413:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.499299:0:8413:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279800 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:2.0:1769074943.499303:0:8413:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.499313:0:8413:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.499331:0:8413:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.499338:0:8413:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549279800/0x7fe2323adfd5611a lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x38:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 36516 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074943.499355:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074943.499357:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074943.499359:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1769074943.499747:0:3650:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491ec380/2, rc 0, transno 17179874653 00000100:00100000:1.0:1769074943.499766:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625289472:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1769074943.503394:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074943.503474:0:6498:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073536 00000100:00100000:0.0:1769074943.503484:0:6498:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073536 00010000:00010000:0.0:1769074943.503527:0:6498:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 8 pid: 36516 timeout: 2031 lvb_type: 0 00000100:00100000:0.0:1769074943.503568:0:6498:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7243 00000100:00100000:0.0:1769074943.503578:0:6498:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617073536:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1769074943.503593:0:6498:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1769074943.503598:0:6498:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x60000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 8 pid: 36516 timeout: 2031 lvb_type: 0 00010000:00010000:0.0:1769074943.503611:0:6498:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x50000400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 8 pid: 36516 timeout: 2031 lvb_type: 0 00010000:00010000:0.0:1769074943.503636:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074943.503638:0:6498:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769074943.503642:0:6498:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.503651:0:6498:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.503676:0:6498:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769074943.503732:0:6498:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617073536:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 155us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074943.503739:0:6498:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7243 00010000:00010000:3.0:1769074943.504406:0:27843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549242200/0x7fe2323adfd55f7d lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->10485759) gid 0 flags: 0x44801400010020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca38 expref: 7 pid: 36516 timeout: 2031 lvb_type: 0 00000100:00100000:3.0:1769074943.504421:0:27843:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:27843:1855007625289856:192.168.204.60@tcp:104: 00010000:00010000:3.0:1769074943.504442:0:27843:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.504463:0:27843:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.504469:0:27843:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.504478:0:27843:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.504488:0:27843:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074943.504511:0:27843:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1769074943.504747:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625289728:12345-0@lo:6:osp-syn-1-1.0 Request processed in 12260us (12507us total) trans 17179874654 rc 0/0 00000100:00100000:3.0:1769074943.504756:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14571 00010000:00010000:0.0:1769074943.504842:0:6502:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1769074943.504849:0:6502:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.504856:0:6502:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279200 already unused, calling callback (ffffffffc0cc1b90) 00010000:00010000:0.0:1769074943.504860:0:6502:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.504868:0:6502:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.504883:0:6502:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.504890:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: ffff89b549279200/0x7fe2323adfd56121 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 27843 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074943.504908:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074943.504910:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074943.504911:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1769074943.505080:0:3649:0:(osp_sync.c:557:osp_sync_interpret()) reply req ffff89b6491edf80/2, rc 0, transno 17179874654 00000100:00100000:1.0:1769074943.505100:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625289728:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1769074943.901156:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625289984:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074943.901231:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074943.901324:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625289984 00000100:00100000:0.0:1769074943.901333:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625289984 00000100:00100000:0.0:1769074943.901385:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1303 00000100:00100000:0.0:1769074943.901395:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625289984:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074943.901417:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3914907648 avail 3683385344 00000100:00100000:0.0:1769074943.901474:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5460eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625289984:12345-0@lo:13:osp-pre-1-1.0 Request processed in 80us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074943.901481:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1303 00000100:00100000:1.0:1769074943.901561:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625289984:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1769074944.478361:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769074945.501553:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074945.949355:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625290112:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.949446:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074945.949462:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625290496:0@lo:400:kworker.0 00000100:00100000:0.0:1769074945.949496:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.949513:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625290624:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.949540:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625290240:0@lo:400:kworker.0 00000100:00100000:3.0:1769074945.949591:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290112 00000100:00100000:3.0:1769074945.949602:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290112 00000100:00100000:3.0:1769074945.949659:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:3.0:1769074945.949668:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625290112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074945.949843:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290496 00000100:00100000:2.0:1769074945.949848:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290496 00000100:00100000:2.0:1769074945.949882:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1645 00000100:00100000:2.0:1769074945.949888:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625290496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.952041:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.952090:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54d1de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625290752:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.952730:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.952758:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625291264:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.952785:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.952844:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1769074945.952852:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625290368:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.952875:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.952903:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625290880:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.952923:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.952940:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625291392:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.952963:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1769074945.953038:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074945.953054:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290240 00000100:00100000:1.0:1769074945.953062:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290240 00000100:00100000:1.0:1769074945.953100:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291264 00000100:00100000:1.0:1769074945.953103:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291264 00000100:00100000:1.0:1769074945.953132:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2800 00000100:00100000:1.0:1769074945.953140:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625290240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953191:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54475bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625290240:12345-0@lo:400:kworker.0 Request processed in 51us (1156us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953198:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2800 00000100:00100000:1.0:1769074945.953217:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2801 00000100:00100000:1.0:1769074945.953230:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625291264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953257:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625291264:12345-0@lo:400:kworker.0 Request processed in 35us (471us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953263:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2801 00000100:00100000:1.0:1769074945.953281:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290752 00000100:00100000:1.0:1769074945.953284:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290752 00000100:00100000:1.0:1769074945.953314:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290368 00000100:00100000:1.0:1769074945.953317:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290368 00000100:00100000:1.0:1769074945.953331:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290880 00000100:00100000:1.0:1769074945.953333:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290880 00000100:00100000:1.0:1769074945.953347:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2591 00000100:00100000:1.0:1769074945.953353:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625290752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953384:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1855007625290752:12345-0@lo:400:kworker.0 Request processed in 31us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953389:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2591 00000100:00100000:1.0:1769074945.953401:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2592 00000100:00100000:1.0:1769074945.953418:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625290368:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953444:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625290368:12345-0@lo:400:kworker.0 Request processed in 27us (568us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953449:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2592 00000100:00100000:1.0:1769074945.953460:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2593 00000100:00100000:1.0:1769074945.953465:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625290880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953488:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b545108380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625290880:12345-0@lo:400:kworker.0 Request processed in 24us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953494:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2593 00000100:00080000:1.0:1769074945.953532:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074945.953534:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074945.953561:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625290368:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953571:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625290880:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953581:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625290240:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953590:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3649:1855007625290752:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953598:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625291264:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953655:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291392 00000100:00100000:1.0:1769074945.953659:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291392 00000100:00100000:1.0:1769074945.953697:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1646 00000100:00100000:1.0:1769074945.953705:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625291392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953739:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625291392:12345-0@lo:400:kworker.0 Request processed in 35us (775us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953744:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1646 00000100:00080000:1.0:1769074945.953757:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074945.953768:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.953790:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625291136:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953839:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.953857:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625291648:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953878:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074945.953903:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291136 00000100:00100000:1.0:1769074945.953906:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291136 00000100:00100000:1.0:1769074945.953929:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291648 00000100:00100000:1.0:1769074945.953932:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291648 00000100:00100000:1.0:1769074945.953949:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2594 00000100:00100000:1.0:1769074945.953954:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625291136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.953989:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d100a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625291136:12345-0@lo:400:kworker.0 Request processed in 35us (149us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.953996:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2594 00000100:00100000:1.0:1769074945.954007:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2595 00000100:00100000:1.0:1769074945.954013:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625291648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.954039:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d103100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625291648:12345-0@lo:400:kworker.0 Request processed in 26us (159us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.954044:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2595 00000100:00080000:1.0:1769074945.954063:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1769074945.954070:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625290624 00000100:00100000:1.0:1769074945.954072:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625290624 00000100:00100000:1.0:1769074945.954095:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1647 00000100:00100000:1.0:1769074945.954100:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625290624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.954127:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d101500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625290624:12345-0@lo:400:kworker.0 Request processed in 27us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.954133:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1647 00000100:00080000:1.0:1769074945.954147:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074945.954163:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54d1dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625290624:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.954171:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625291136:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.954181:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625291648:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.954207:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625291392:0@lo:400:kworker.0 00000100:00100000:0.0:1769074945.959554:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625291008:0@lo:400:kworker.0 00000100:00100000:0.0:1769074945.959626:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074945.959651:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625291520:0@lo:400:kworker.0 00000100:00100000:0.0:1769074945.960534:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074945.960602:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291520 00000100:00100000:0.0:1769074945.960610:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291520 00000100:00100000:0.0:1769074945.960665:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1648 00000100:00100000:0.0:1769074945.960673:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625291520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769074945.960740:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625291520:12345-0@lo:400:kworker.0 Request processed in 67us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074945.960748:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1648 00000100:00080000:0.0:1769074945.960774:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769074945.960833:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625291520:0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.962856:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291008 00000100:00100000:1.0:1769074945.962874:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291008 00000100:00100000:1.0:1769074945.962955:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2596 00000100:00100000:1.0:1769074945.962965:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625291008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074945.963067:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1855007625291008:12345-0@lo:400:kworker.0 Request processed in 103us (3440us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074945.963074:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2596 00000100:00080000:0.0:1769074945.963877:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769074945.963934:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3651:1855007625291008:0@lo:400:kworker.0 00000100:00100000:3.0:1769074945.965329:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625290112:12345-0@lo:400:kworker.0 Request processed in 15655us (15876us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074945.965339:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625290112:0@lo:400:kworker.0 00000100:00100000:3.0:1769074945.965343:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:3.0:1769074945.975434:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625290496:12345-0@lo:400:kworker.0 Request processed in 25538us (25929us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074945.975450:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1645 00000100:00100000:0.0:1769074945.975475:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625290496:0@lo:400:kworker.0 00010000:00010000:1.0:1769074946.531505:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074947.014101:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074947.014164:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074947.014207:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074947.014230:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074947.014259:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074947.014290:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073792 00000100:00100000:1.0:1769074947.014299:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073792 00000100:00100000:1.0:1769074947.014353:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073920 00000100:00100000:1.0:1769074947.014356:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073920 00000100:00100000:1.0:1769074947.014381:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2597 00000100:00100000:1.0:1769074947.014420:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007617073792:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074947.014504:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007617073792:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 86us (414us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074947.014513:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2597 00000100:00100000:1.0:1769074947.014532:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2598 00000100:00100000:1.0:1769074947.014538:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617073920:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074947.014569:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617073920:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 31us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074947.014576:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2598 00000100:00100000:1.0:1769074947.014594:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074048 00000100:00100000:1.0:1769074947.014598:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074048 00000100:00100000:1.0:1769074947.014620:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074176 00000100:00100000:1.0:1769074947.014623:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074176 00000100:00100000:1.0:1769074947.014639:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1649 00000100:00100000:1.0:1769074947.014645:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617074048:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074947.014672:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617074048:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (507us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074947.014678:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1649 00000100:00100000:1.0:1769074947.014690:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1650 00000100:00100000:1.0:1769074947.014695:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617074176:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074947.014732:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617074176:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 37us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074947.014738:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1650 00000100:00100000:3.0:1769074947.015688:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617073664 00000100:00100000:3.0:1769074947.015698:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617073664 00000100:00100000:3.0:1769074947.015755:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 411 00000100:00100000:3.0:1769074947.015764:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617073664:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074947.015854:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617073664:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 91us (1657us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074947.015861:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 411 00000100:00080000:0.0:1769074947.016969:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074947.016976:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074947.016980:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074947.016983:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1769074947.549459:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074947.999209:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074947.999234:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625291776:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074947.999302:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074947.999355:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291776 00000100:00100000:3.0:1769074947.999363:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291776 00000100:00100000:3.0:1769074947.999437:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2802 00000100:00100000:3.0:1769074947.999446:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625291776:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074947.999486:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:3.0:1769074947.999537:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1855007625291776:12345-0@lo:41:osp-pre-0-1.0 Request processed in 91us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074947.999544:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2802 00000100:00100000:3.0:1769074947.999617:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625291776:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074948.000482:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769074948.000496:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625291904:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074948.000538:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074948.000735:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625291904 00000100:00100000:3.0:1769074948.000741:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625291904 00000100:00100000:3.0:1769074948.000783:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2803 00000100:00100000:3.0:1769074948.000791:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625291904:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074948.000835:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:3.0:1769074948.000882:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625291904:12345-0@lo:41:osp-pre-1-0.0 Request processed in 91us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074948.000889:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2803 00000100:00100000:0.0:1769074948.000978:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625291904:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074948.509015:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625292032:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074948.509084:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074948.509235:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292032 00000100:00100000:2.0:1769074948.509244:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292032 00000100:00100000:2.0:1769074948.509306:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1304 00000100:00100000:2.0:1769074948.509315:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625292032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074948.509343:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074948.509398:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547599f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625292032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074948.509406:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1304 00000100:00100000:1.0:1769074948.509540:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625292160:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074948.509568:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074948.509677:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074948.509685:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625292288:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074948.509713:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074948.509736:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292160 00000100:00100000:2.0:1769074948.509740:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292160 00000100:00100000:2.0:1769074948.509773:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292288 00000100:00100000:2.0:1769074948.509777:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292288 00000100:00100000:2.0:1769074948.509797:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1305 00000100:00100000:2.0:1769074948.509803:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625292160:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1769074948.509833:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074948.509863:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625292160:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074948.509869:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1305 00000100:00100000:2.0:1769074948.509882:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1306 00000100:00100000:2.0:1769074948.509888:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625292288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074948.509902:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074948.509931:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625292288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074948.509937:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1306 00000100:00100000:2.0:1769074948.509981:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625292288:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074948.510006:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625292160:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1769074948.510257:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769074948.510262:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074948.510288:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625292032:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1769074948.573159:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074949.022312:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1769074949.022338:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625292416:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074949.022449:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074949.022603:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292416 00000100:00100000:3.0:1769074949.022613:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292416 00000100:00100000:3.0:1769074949.022734:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1307 00000100:00100000:3.0:1769074949.022744:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625292416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074949.022767:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074949.022854:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6491ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+7:3649:x1855007625292416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 112us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074949.022863:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1307 00000100:00080000:0.0:1769074949.023162:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074949.023247:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625292416:0@lo:13:osp-pre-1-1.0 00000001:00080000:3.0:1769074949.028530:0:26036:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 17179874654 is committed 00000001:00080000:0.0:1769074949.028643:0:24714:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 21474841323 is committed 00010000:00010000:1.0:1769074949.600042:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769074950.624913:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074951.069672:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625292672:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.069771:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.069851:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625293184:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.069878:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.069894:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625293696:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.069912:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.069934:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:3.0:1769074951.069939:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625292544:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.069962:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.069980:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625293056:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.070001:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.070020:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625293568:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.070040:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.070054:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5499d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625294080:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.070078:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074951.070105:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292672 00000100:00100000:3.0:1769074951.070113:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292672 00000100:00100000:3.0:1769074951.070166:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293696 00000100:00100000:3.0:1769074951.070169:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293696 00000100:00100000:1.0:1769074951.070722:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625292800:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.070761:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074951.070784:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625293312:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.070833:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074951.070851:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b427800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625293824:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.070871:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074951.070891:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1769074951.070897:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547599500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625292928:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.070916:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074951.070932:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625293440:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.070949:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074951.070959:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625293952:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.070976:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074951.071026:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292544 00000100:00100000:1.0:1769074951.071031:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292544 00000100:00100000:1.0:1769074951.071066:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1769074951.071074:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625292544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071127:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3652:x1855007625292544:12345-0@lo:400:kworker.0 Request processed in 53us (1164us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071147:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1769074951.071194:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293184 00000100:00100000:1.0:1769074951.071198:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293184 00000100:00100000:1.0:1769074951.071227:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293568 00000100:00100000:1.0:1769074951.071230:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293568 00000100:00100000:1.0:1769074951.071244:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294080 00000100:00100000:1.0:1769074951.071247:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294080 00000100:00100000:1.0:1769074951.071258:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292800 00000100:00100000:1.0:1769074951.071260:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292800 00000100:00100000:1.0:1769074951.071268:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293312 00000100:00100000:1.0:1769074951.071271:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293312 00000100:00100000:1.0:1769074951.071281:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293440 00000100:00100000:1.0:1769074951.071283:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293440 00000100:00100000:1.0:1769074951.071297:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2599 00000100:00100000:1.0:1769074951.071303:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625293184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071347:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a273100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1855007625293184:12345-0@lo:400:kworker.0 Request processed in 44us (1468us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071353:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2599 00000100:00100000:1.0:1769074951.071366:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2600 00000100:00100000:1.0:1769074951.071372:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625293568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071397:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1855007625293568:12345-0@lo:400:kworker.0 Request processed in 25us (1356us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071402:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2600 00000100:00100000:1.0:1769074951.071416:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2601 00000100:00100000:1.0:1769074951.071421:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625294080:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071465:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1855007625294080:12345-0@lo:400:kworker.0 Request processed in 43us (1385us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071470:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2601 00000100:00100000:1.0:1769074951.071483:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2602 00000100:00100000:1.0:1769074951.071488:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625292800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071520:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3649:x1855007625292800:12345-0@lo:400:kworker.0 Request processed in 32us (759us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071526:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2602 00000100:00100000:1.0:1769074951.071537:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2603 00000100:00100000:1.0:1769074951.071543:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625293312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071568:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf68380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1855007625293312:12345-0@lo:400:kworker.0 Request processed in 26us (735us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071574:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2603 00000100:00100000:1.0:1769074951.071584:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2604 00000100:00100000:1.0:1769074951.071589:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625293440:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071613:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625293440:12345-0@lo:400:kworker.0 Request processed in 25us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071619:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2604 00000100:00100000:1.0:1769074951.071642:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2804 00000100:00100000:1.0:1769074951.071649:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625292672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071680:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625292672:12345-0@lo:400:kworker.0 Request processed in 32us (1910us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071686:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2804 00000100:00100000:1.0:1769074951.071713:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293056 00000100:00100000:1.0:1769074951.071718:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293056 00000100:00100000:1.0:1769074951.071768:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293824 00000100:00100000:1.0:1769074951.071771:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293824 00000100:00100000:1.0:1769074951.071828:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625292928 00000100:00100000:1.0:1769074951.071831:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625292928 00000100:00100000:1.0:1769074951.071868:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625293952 00000100:00100000:1.0:1769074951.071872:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625293952 00000100:00100000:1.0:1769074951.071889:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1651 00000100:00100000:1.0:1769074951.071894:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625293056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071925:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625293056:12345-0@lo:400:kworker.0 Request processed in 31us (1923us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071931:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1651 00000100:00100000:1.0:1769074951.071942:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1652 00000100:00100000:1.0:1769074951.071948:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625293824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.071974:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625293824:12345-0@lo:400:kworker.0 Request processed in 26us (1102us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.071979:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1652 00000100:00100000:1.0:1769074951.071989:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1653 00000100:00100000:1.0:1769074951.071994:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625292928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072017:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54cf6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625292928:12345-0@lo:400:kworker.0 Request processed in 22us (1099us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.072022:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1653 00000100:00100000:1.0:1769074951.072031:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1654 00000100:00100000:1.0:1769074951.072036:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625293952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072056:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5461e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625293952:12345-0@lo:400:kworker.0 Request processed in 20us (1079us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074951.072061:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1654 00000100:00100000:1.0:1769074951.072116:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:025c5dec-f591-4c93-9b92-fac36a3bdc36:3652:1855007625292544:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072128:0:3652:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 17179874654 gen 2 00000004:00080000:1.0:1769074951.072133:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491ec380, transno 17179874653 00000004:00080000:1.0:1769074951.072142:0:3652:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b6491edf80, transno 17179874654 00000100:00100000:1.0:1769074951.072146:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625293056:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072156:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3652:1855007625293568:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072165:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5499d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3652:1855007625294080:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072174:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625292672:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072186:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3652:1855007625293184:0@lo:400:kworker.0 00000100:00080000:1.0:1769074951.072222:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074951.072226:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074951.072229:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074951.072231:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074951.072243:0:3649:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 21474841323 gen 2 00000004:00080000:1.0:1769074951.072247:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c69880, transno 21474841322 00000004:00080000:1.0:1769074951.072254:0:3649:0:(osp_sync.c:505:osp_sync_request_commit_cb()) commit req ffff89b546c69c00, transno 21474841323 00000100:00100000:1.0:1769074951.072258:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547599500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625292928:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072267:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625293440:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072275:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b426300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625293952:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072283:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3649:1855007625292800:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072292:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3649:1855007625293312:0@lo:400:kworker.0 00000100:00100000:1.0:1769074951.072300:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54b427800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625293824:0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.072987:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2805 00000100:00100000:3.0:1769074951.073000:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625293696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074951.073052:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a271180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625293696:12345-0@lo:400:kworker.0 Request processed in 57us (3138us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074951.073058:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2805 00000100:00100000:1.0:1769074951.077841:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625293696:0@lo:400:kworker.0 00010000:00010000:2.0:1769074951.644988:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074952.132587:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074952.132635:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074952.132676:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074952.132710:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074952.132739:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074952.132771:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074560 00000100:00100000:1.0:1769074952.132781:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074560 00000100:00100000:1.0:1769074952.132888:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074432 00000100:00100000:1.0:1769074952.132892:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074432 00000100:00100000:1.0:1769074952.132921:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2605 00000100:00100000:1.0:1769074952.133098:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617074560:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074952.133185:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617074560:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 254us (548us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074952.133193:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2605 00000100:00100000:1.0:1769074952.133214:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2606 00000100:00100000:1.0:1769074952.133222:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007617074432:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074952.133249:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5499d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2406:x1855007617074432:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 28us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074952.133255:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2606 00000100:00080000:1.0:1769074952.133275:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074952.133278:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1769074952.133501:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074816 00000100:00100000:0.0:1769074952.133507:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074816 00000100:00100000:0.0:1769074952.133560:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074688 00000100:00100000:0.0:1769074952.133564:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074688 00000100:00100000:0.0:1769074952.133598:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1655 00000100:00100000:0.0:1769074952.133605:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617074816:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074952.133643:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b426300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2406:x1855007617074816:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 39us (967us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074952.133649:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1655 00000100:00100000:0.0:1769074952.133661:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1656 00000100:00100000:0.0:1769074952.133666:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617074688:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074952.133689:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2406:x1855007617074688:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 23us (950us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074952.133694:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1656 00000100:00100000:3.0:1769074952.134860:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074304 00000100:00100000:3.0:1769074952.134870:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074304 00000100:00100000:3.0:1769074952.134941:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 413 00000100:00100000:3.0:1769074952.134951:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617074304:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074952.135005:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2406:x1855007617074304:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 56us (2431us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074952.135014:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 413 00000100:00080000:1.0:1769074952.135247:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074952.135253:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1769074952.669021:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074953.117084:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1769074953.121901:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625294208:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074953.121983:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074953.122034:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294208 00000100:00100000:0.0:1769074953.122042:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294208 00000100:00100000:0.0:1769074953.122091:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2806 00000100:00100000:0.0:1769074953.122112:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625294208:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1769074953.122148:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:0.0:1769074953.122194:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625294208:12345-0@lo:41:osp-pre-0-1.0 Request processed in 95us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074953.122202:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2806 00000100:00100000:0.0:1769074953.122255:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625294208:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1769074953.122964:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625294336:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074953.123003:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074953.123185:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294336 00000100:00100000:1.0:1769074953.123191:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294336 00000100:00100000:1.0:1769074953.123230:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2807 00000100:00100000:1.0:1769074953.123238:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625294336:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1769074953.123260:0:35531:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:1.0:1769074953.123313:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625294336:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074953.123319:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2807 00000100:00100000:3.0:1769074953.128887:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625294336:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074953.630045:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074953.630078:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625294592:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074953.630157:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074953.630220:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294592 00000100:00100000:1.0:1769074953.630228:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294592 00000100:00100000:1.0:1769074953.630292:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1308 00000100:00100000:1.0:1769074953.630310:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625294592:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1769074953.630339:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:1.0:1769074953.630382:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625294592:12345-0@lo:13:osp-pre-0-1.0 Request processed in 83us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074953.630390:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1308 00000100:00080000:1.0:1769074953.630417:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769074953.630453:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5436d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625294592:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074953.630963:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625294464:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769074953.631001:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074953.631126:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074953.631135:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625294720:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074953.631159:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074953.631215:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294464 00000100:00100000:1.0:1769074953.631220:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294464 00000100:00100000:1.0:1769074953.631253:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294720 00000100:00100000:1.0:1769074953.631255:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294720 00000100:00100000:1.0:1769074953.631268:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1309 00000100:00100000:1.0:1769074953.631282:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625294464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1769074953.631296:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00080000:2.0:1769074953.631327:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769074953.631331:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1769074953.631549:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1310 00000100:00100000:3.0:1769074953.631559:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b849850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625294720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769074953.631575:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074953.631611:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b849850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625294720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (448us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074953.631617:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1310 00000100:00100000:3.0:1769074953.631669:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625294720:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769074953.631848:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54759c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625294464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 573us (844us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074953.631854:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1309 00000100:00100000:2.0:1769074953.635178:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625294464:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1769074953.693572:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074954.144128:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625294848:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074954.144198:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074954.146202:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294848 00000100:00100000:0.0:1769074954.146216:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294848 00000100:00100000:0.0:1769074954.146340:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1311 00000100:00100000:0.0:1769074954.146351:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625294848:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074954.146376:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074954.146451:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625294848:12345-0@lo:13:osp-pre-1-1.0 Request processed in 102us (2251us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074954.146473:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1311 00000100:00100000:0.0:1769074954.146904:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625294848:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1769074954.147226:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1769074954.717431:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000004:3.0:1769074955.186632:0:39810:0:(tgt_mount.c:2194:server_getattr()) lustre-OST0001: root_inode from vfsmnt ino=2, dev=0 00000020:00000004:3.0:1769074955.186642:0:39810:0:(tgt_mount.c:2225:is_cmd_supported()) ioctl cmd=c0185879 00000020:00000004:3.0:1769074955.224380:0:39817:0:(tgt_mount.c:2194:server_getattr()) lustre-OST0000: root_inode from vfsmnt ino=2, dev=0 00000020:00000004:3.0:1769074955.224392:0:39817:0:(tgt_mount.c:2225:is_cmd_supported()) ioctl cmd=c0185879 00010000:00010000:1.0:1769074955.741075:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074956.190004:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625295360:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190085:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190186:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625295872:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190214:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190244:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625296384:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190289:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190337:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:3.0:1769074956.190343:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625294976:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190370:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190407:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625295488:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190431:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190448:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625296000:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190468:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190486:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625296512:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190505:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190530:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625295104:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190553:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190570:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625295616:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190603:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190620:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625296128:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190640:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190658:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1769074956.190664:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54759c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625295232:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190681:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190694:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625295744:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190711:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190720:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625296256:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190737:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074956.190769:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295360 00000100:00100000:3.0:1769074956.190777:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295360 00000100:00100000:3.0:1769074956.190845:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296384 00000100:00100000:3.0:1769074956.190849:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296384 00000100:00100000:3.0:1769074956.190872:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295488 00000100:00100000:3.0:1769074956.190874:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295488 00000100:00100000:3.0:1769074956.190913:0:8398:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296256 00000100:00100000:3.0:1769074956.190916:0:8398:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296256 00000100:00100000:3.0:1769074956.190938:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1657 00000100:00100000:3.0:1769074956.190945:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625295360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.190992:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625295360:12345-0@lo:400:kworker.0 Request processed in 47us (907us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.190998:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1657 00000100:00100000:3.0:1769074956.191012:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1658 00000100:00100000:3.0:1769074956.191018:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625296384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191042:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625296384:12345-0@lo:400:kworker.0 Request processed in 25us (753us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191047:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1658 00000100:00100000:3.0:1769074956.191056:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1659 00000100:00100000:3.0:1769074956.191061:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625295488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191084:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625295488:12345-0@lo:400:kworker.0 Request processed in 23us (653us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191089:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1659 00000100:00100000:3.0:1769074956.191098:0:8398:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1660 00000100:00100000:3.0:1769074956.191103:0:8398:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625296256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191130:0:8398:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625296256:12345-0@lo:400:kworker.0 Request processed in 26us (391us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191135:0:8398:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1660 00000100:00100000:3.0:1769074956.191152:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295872 00000100:00100000:3.0:1769074956.191156:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295872 00000100:00100000:3.0:1769074956.191174:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296000 00000100:00100000:3.0:1769074956.191177:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296000 00000100:00100000:3.0:1769074956.191186:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296512 00000100:00100000:3.0:1769074956.191188:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296512 00000100:00100000:3.0:1769074956.191197:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295616 00000100:00100000:3.0:1769074956.191199:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295616 00000100:00100000:3.0:1769074956.191208:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295232 00000100:00100000:3.0:1769074956.191210:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295232 00000100:00100000:3.0:1769074956.191218:0:6513:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295744 00000100:00100000:3.0:1769074956.191220:0:6513:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295744 00000100:00100000:3.0:1769074956.191243:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2607 00000100:00100000:3.0:1769074956.191250:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625295872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191281:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c69880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625295872:12345-0@lo:400:kworker.0 Request processed in 32us (1067us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191287:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2607 00000100:00100000:3.0:1769074956.191299:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2608 00000100:00100000:3.0:1769074956.191304:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625296000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191329:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1855007625296000:12345-0@lo:400:kworker.0 Request processed in 25us (860us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191335:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2608 00000100:00100000:3.0:1769074956.191345:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2609 00000100:00100000:3.0:1769074956.191350:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625296512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191375:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1855007625296512:12345-0@lo:400:kworker.0 Request processed in 25us (869us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191381:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2609 00000100:00100000:3.0:1769074956.191391:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2610 00000100:00100000:3.0:1769074956.191397:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625295616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191420:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625295616:12345-0@lo:400:kworker.0 Request processed in 23us (816us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191424:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2610 00000100:00100000:3.0:1769074956.191435:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2611 00000100:00100000:3.0:1769074956.191440:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625295232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191463:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625295232:12345-0@lo:400:kworker.0 Request processed in 23us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191468:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2611 00000100:00100000:3.0:1769074956.191478:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2612 00000100:00100000:3.0:1769074956.191483:0:6513:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625295744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191527:0:6513:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625295744:12345-0@lo:400:kworker.0 Request processed in 43us (814us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191532:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2612 00000100:00100000:3.0:1769074956.191570:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625294976 00000100:00100000:3.0:1769074956.191574:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625294976 00000100:00100000:3.0:1769074956.191599:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1769074956.191604:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625294976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191647:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3649:x1855007625294976:12345-0@lo:400:kworker.0 Request processed in 42us (1275us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191652:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1769074956.191683:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625295104 00000100:00100000:3.0:1769074956.191687:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625295104 00000100:00100000:3.0:1769074956.191708:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296128 00000100:00100000:3.0:1769074956.191711:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296128 00000100:00100000:3.0:1769074956.191726:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2808 00000100:00100000:3.0:1769074956.191732:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625295104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191761:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67ff0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625295104:12345-0@lo:400:kworker.0 Request processed in 29us (1207us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191766:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2808 00000100:00100000:3.0:1769074956.191778:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2809 00000100:00100000:3.0:1769074956.191783:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625296128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191830:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625296128:12345-0@lo:400:kworker.0 Request processed in 47us (1189us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074956.191836:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2809 00000100:00100000:3.0:1769074956.191887:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625295232:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191897:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625295744:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191906:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625296256:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191914:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625295104:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191921:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625295616:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191929:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c68000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625296128:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191972:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:025c5dec-f591-4c93-9b92-fac36a3bdc36:3649:1855007625294976:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191982:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625295488:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191991:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3649:1855007625296000:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.191999:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3649:1855007625296512:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.192008:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54759c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625295360:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.192017:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625295872:0@lo:400:kworker.0 00000100:00100000:3.0:1769074956.192026:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625296384:0@lo:400:kworker.0 00000100:00080000:0.0:1769074956.193404:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074956.193412:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074956.193415:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769074956.193418:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1769074956.766982:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074957.250985:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074957.251031:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074957.251058:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074957.251082:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074957.251106:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:1.0:1769074957.251135:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075456 00000100:00100000:1.0:1769074957.251145:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075456 00000100:00100000:1.0:1769074957.251195:0:28955:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075328 00000100:00100000:1.0:1769074957.251198:0:28955:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075328 00000100:00100000:1.0:1769074957.251248:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1661 00000100:00100000:1.0:1769074957.251259:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617075456:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074957.251327:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617075456:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 71us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074957.251335:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1661 00000100:00100000:1.0:1769074957.251355:0:28955:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1662 00000100:00100000:1.0:1769074957.251362:0:28955:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617075328:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:1.0:1769074957.251387:0:28955:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2407:x1855007617075328:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 26us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074957.251393:0:28955:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1662 00000100:00100000:0.0:1769074957.251781:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075200 00000100:00100000:0.0:1769074957.251787:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075200 00000100:00100000:0.0:1769074957.251849:0:6514:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075072 00000100:00100000:0.0:1769074957.251853:0:6514:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075072 00000100:00100000:0.0:1769074957.251879:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2613 00000100:00100000:0.0:1769074957.251887:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617075200:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074957.251929:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2407:x1855007617075200:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 42us (897us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074957.251936:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2613 00000100:00100000:0.0:1769074957.251949:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2614 00000100:00100000:0.0:1769074957.251956:0:6514:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007617075072:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074957.251980:0:6514:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5436d4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:2407:x1855007617075072:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (898us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074957.251986:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2614 00000100:00080000:0.0:1769074957.252007:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769074957.252011:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1769074957.252014:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1769074957.252016:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074957.252183:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617074944 00000100:00100000:3.0:1769074957.252189:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617074944 00000100:00100000:3.0:1769074957.252273:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 415 00000100:00100000:3.0:1769074957.252281:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67a08e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617074944:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074957.252314:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67a08e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2407:x1855007617074944:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 34us (1338us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074957.252321:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 415 00010000:00010000:0.0:1769074957.789181:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074958.237424:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769074958.237459:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625296640:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074958.237559:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074958.237689:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296640 00000100:00100000:3.0:1769074958.237701:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296640 00000100:00100000:3.0:1769074958.237776:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2810 00000100:00100000:3.0:1769074958.237788:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625296640:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1769074958.237870:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445355520 avail 1299116032 00000100:00100000:3.0:1769074958.238098:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b410000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1855007625296640:12345-0@lo:41:osp-pre-0-1.0 Request processed in 310us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074958.238109:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2810 00000100:00100000:2.0:1769074958.238216:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625296640:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074958.238562:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625296768:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074958.238598:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074958.238773:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296768 00000100:00100000:3.0:1769074958.238779:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296768 00000100:00100000:3.0:1769074958.238873:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2811 00000100:00100000:3.0:1769074958.238894:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625296768:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1769074958.238921:0:6517:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446121472 avail 1299881984 00000100:00100000:3.0:1769074958.243265:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b413800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625296768:12345-0@lo:41:osp-pre-1-0.0 Request processed in 4355us (4649us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074958.243283:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2811 00000100:00100000:2.0:1769074958.243453:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625296768:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1769074958.749653:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546f92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625297024:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074958.749683:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769074958.749715:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625297152:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074958.749745:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074958.749838:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1769074958.749846:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625296896:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074958.749872:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074958.749898:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074958.750140:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297024 00000100:00100000:2.0:1769074958.750149:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297024 00000100:00080000:1.0:1769074958.750214:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074958.750241:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625296896 00000100:00100000:2.0:1769074958.750246:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625296896 00000100:00100000:1.0:1769074958.750259:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297152 00000100:00100000:1.0:1769074958.750265:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297152 00000100:00100000:2.0:1769074958.750277:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1312 00000100:00100000:2.0:1769074958.750289:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625297024:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1769074958.750298:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1313 00000100:00100000:1.0:1769074958.750308:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625296896:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1769074958.750319:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00002000:00000024:1.0:1769074958.750328:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:2.0:1769074958.750367:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625297024:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (624us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074958.750375:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1312 00000100:00100000:2.0:1769074958.750392:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1314 00000100:00100000:2.0:1769074958.750399:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625297152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769074958.750409:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:2.0:1769074958.750439:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625297152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 40us (542us total) trans 0 rc 0/0 00000100:00080000:0.0:1769074958.750446:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074958.750446:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1314 00000100:00080000:0.0:1769074958.750453:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1769074958.751108:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546f92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625297024:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1769074958.757544:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625297152:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769074958.757976:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b412300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625296896:12345-0@lo:13:osp-pre-1-0.0 Request processed in 7667us (8099us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074958.757989:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1313 00000100:00100000:2.0:1769074958.758023:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b549f28e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625296896:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1769074958.813255:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074959.261295:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625297280:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074959.261373:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074959.261660:0:12151:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297280 00000100:00100000:3.0:1769074959.261669:0:12151:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297280 00000100:00100000:3.0:1769074959.261736:0:12151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1315 00000100:00100000:3.0:1769074959.261745:0:12151:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625297280:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1769074959.261766:0:12151:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:3.0:1769074959.261862:0:12151:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625297280:12345-0@lo:13:osp-pre-1-1.0 Request processed in 118us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074959.261870:0:12151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1315 00000100:00080000:0.0:1769074959.262006:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074959.262592:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625297280:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1769074959.433074:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074959.433214:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075584 00000100:00100000:0.0:1769074959.433225:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075584 00000100:00100000:0.0:1769074959.433317:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13687 00000100:00100000:0.0:1769074959.433328:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:69555:x1855007617075584:12345-192.168.204.60@tcp:41:stat.0 00000004:00000024:0.0:1769074959.433390:0:6512:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7830949888 avail 7367905280 00000100:00100000:0.0:1769074959.433473:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b425180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:69555:x1855007617075584:12345-192.168.204.60@tcp:41:stat.0 Request processed in 147us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074959.433481:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13687 00000100:00080000:1.0:1769074959.433786:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1769074959.495846:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.495954:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075712 00000100:00100000:2.0:1769074959.495964:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075712 00000100:00100000:2.0:1769074959.496037:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13688 00000100:00100000:2.0:1769074959.496047:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:69559:x1855007617075712:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074959.496069:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074959.496090:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd56128 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879ca9a expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.496155:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496169:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496191:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496243:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496250:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496257:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496327:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496334:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496340:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496347:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496352:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88c00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.496356:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496362:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496375:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074959.496378:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.496384:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496389:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5612f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496404:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496410:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496416:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496420:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.496423:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496428:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496434:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa25ad81b (ffff89b65436a000) 00010000:00010000:2.0:1769074959.496436:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.496438:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496443:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56136 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.496472:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd56128 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca9a expref: 25 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.496482:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074959.496484:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.496488:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd56128 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879ca9a expref: 25 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.496513:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89e00, rc 0) 00000100:00100000:2.0:1769074959.497043:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54b427b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:69559:x1855007617075712:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 995us (1206us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074959.497051:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13688 00000100:00100000:3.0:1769074959.513549:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.513986:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617075968 00000100:00100000:2.0:1769074959.513997:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617075968 00000100:00100000:2.0:1769074959.514063:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13689 00000100:00100000:2.0:1769074959.514074:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+24:69559:x1855007617075968:12345-192.168.204.60@tcp:36:lfs.0 00010000:00010000:2.0:1769074959.514134:0:8855:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074959.514164:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b67febaa00/0x7fe2323adfd55d0e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879c974 expref: 24 pid: 6514 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.514196:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074959.514201:0:8855:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769074959.514263:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.514277:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.514287:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.514302:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56144 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.514308:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56144 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.514317:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56144 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000040:00080000:2.0:1769074959.517486:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].88, 2168 off97144 00000040:00080000:2.0:1769074959.519485:0:8855:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000405:0x2:0x0].10, 2368 off54464 00000004:00080000:2.0:1769074959.519500:0:8855:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff89b66e6613c0 with version 25 00000004:00080000:1.0:1769074959.519683:0:32392:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff89b650039b40 version 25 rpc_version 25 00000020:00100000:1.0:1769074959.519726:0:32392:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1769074959.519739:0:32392:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff89b5437b5180 x1855007625297536/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1769074959.519768:0:32392:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5437b5180 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625297536:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1769074959.519881:0:32392:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074959.519945:0:32392:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b67b452b40 going to sleep for 11 seconds 00000100:00100000:3.0:1769074959.520361:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297536 00000100:00100000:3.0:1769074959.520371:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297536 00000100:00100000:3.0:1769074959.520422:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2812 00000100:00100000:3.0:1769074959.520431:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:32392:x1855007625297536:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1769074959.521642:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:32392:x1855007625297536:12345-0@lo:1000:osp_up1-0.0 Request processed in 1208us (1756us total) trans 4294968392 rc 0/0 00000100:00100000:3.0:1769074959.521653:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2812 00000100:00100000:1.0:1769074959.521877:0:32392:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5437b5180 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:32392:1855007625297536:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1769074959.522056:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5614b lrc: 3/0,1 mode: --/PW res: [0x200000405:0x20d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522074:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5614b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522085:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5614b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522105:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd56152 lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522111:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd56152 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522117:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd56152 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522151:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522157:0:8855:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.522167:0:8855:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 3/0,1 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8855 initiator: MDT0 00000100:00100000:2.0:1769074959.522194:0:8855:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:8855:1855007625297664:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1769074959.522230:0:8855:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074959.522259:0:8855:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b549266d80 going to sleep for 11 seconds 00000100:00100000:0.0:1769074959.522314:0:35531:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297664 00000100:00100000:0.0:1769074959.522322:0:35531:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297664 00000100:00100000:0.0:1769074959.522372:0:35531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2813 00000100:00100000:0.0:1769074959.522380:0:35531:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:8855:x1855007625297664:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1769074959.522399:0:35531:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074959.522430:0:35531:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 2/0,0 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7fe2323adfd56159 expref: -99 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.522451:0:35531:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7fe2323adfd56159 expref: 7 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.522464:0:35531:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd56159 expref: 7 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.522473:0:35531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xb:0x0].0x0 (ffff89b67a317900) 00010000:00010000:0.0:1769074959.522476:0:35531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074959.522480:0:35531:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135c00, rc 0) 00000100:00100000:0.0:1769074959.522524:0:35531:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b67b411500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:8855:x1855007625297664:12345-0@lo:101:mdt00_003.0 Request processed in 144us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074959.522530:0:35531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2813 00000100:00100000:2.0:1769074959.522956:0:8855:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:8855:1855007625297664:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1769074959.522969:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.522978:0:8855:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1769074959.523008:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000d70b93bf reply state 000000001ec2e235 transno 8589944145 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd56152 lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523020:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88e00/0x7fe2323adfd56152 lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523025:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:2.0:1769074959.523027:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074959.523030:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769074959.523035:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 4/0,1 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.523043:0:8855:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1769074959.523051:0:8855:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000405:0xb:0x0] 00000004:00010000:2.0:1769074959.523060:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000d70b93bf reply state 000000001ec2e235 transno 8589944145 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5614b lrc: 3/0,1 mode: PW/PW res: [0x200000405:0x20d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523066:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88c00/0x7fe2323adfd5614b lrc: 3/0,1 mode: TXN/TXN res: [0x200000405:0x20d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523072:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20d:0x0].0x0 (ffff89b65436a800) 00010000:00010000:2.0:1769074959.523074:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074959.523076:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1769074959.523081:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20d:0x0]: returning size 4096 00010000:00010000:2.0:1769074959.523090:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523095:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523100:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523106:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523111:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89e00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.523115:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523122:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523130:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074959.523133:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.523136:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523142:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd5613d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000004:00010000:2.0:1769074959.523154:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 00000000d70b93bf reply state 000000001ec2e235 transno 8589944145 ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56144 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523162:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88000/0x7fe2323adfd56144 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.523171:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa25ad81b (ffff89b65436a000) 00010000:00010000:2.0:1769074959.523184:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1769074959.523186:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1769074959.523256:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f28e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:69559:x1855007617075968:12345-192.168.204.60@tcp:36:lfs.0 Request processed in 9183us (9735us total) trans 8589944145 rc 0/0 00000100:00100000:2.0:1769074959.523264:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13689 00000100:00100000:1.0:1769074959.555259:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.555481:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076096 00000100:00100000:2.0:1769074959.555492:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076096 00000100:00100000:2.0:1769074959.555715:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13690 00000100:00100000:2.0:1769074959.555727:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+25:69562:x1855007617076096:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074959.555751:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074959.555773:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd56167 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879caa1 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.555962:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5616e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.555976:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5616e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.555986:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5616e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074959.556030:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1769074959.556033:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1769074959.556155:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5616e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.556258:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89e00/0x7fe2323adfd56167 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caa1 expref: 27 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.556333:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89200/0x7fe2323adfd5616e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caa1 expref: 26 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.556343:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074959.556346:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769074959.556350:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b666b89200, rc 0) 00000100:00100000:2.0:1769074959.556931:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:69562:x1855007617076096:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1205us (1680us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074959.556940:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13690 00000100:00100000:1.0:1769074959.561393:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.562153:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076224 00000100:00100000:2.0:1769074959.562164:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076224 00000100:00100000:2.0:1769074959.564386:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13691 00000100:00100000:2.0:1769074959.564404:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+26:69562:x1855007617076224:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074959.564425:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074959.564449:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd56175 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879caa8 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.564514:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564646:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564660:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564673:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564684:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564690:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564769:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46c00/0x7fe2323adfd5618a lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564778:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46c00/0x7fe2323adfd5618a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564785:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46c00/0x7fe2323adfd5618a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074959.564898:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20d:0x0]: returning size 4096 00000004:00200000:2.0:1769074959.564903:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1769074959.564922:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46c00/0x7fe2323adfd5618a lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564933:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564940:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564946:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564952:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564958:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.564962:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564969:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564981:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:2.0:1769074959.564983:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.564987:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.564993:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89a00/0x7fe2323adfd5617c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.565008:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.565014:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.565019:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.565024:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b88800 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.565027:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.565032:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.566471:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa25ad81b (ffff89b65436a000) 00010000:00010000:2.0:1769074959.566476:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.566481:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.566489:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b88800/0x7fe2323adfd56183 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa25ad81b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.566886:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b666b89800/0x7fe2323adfd56175 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caa8 expref: 28 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.566903:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46c00/0x7fe2323adfd5618a lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caa8 expref: 27 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.566911:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20d:0x0].0x0 (ffff89b65436a800) 00010000:00010000:2.0:1769074959.566913:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1769074959.566916:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46c00, rc 0) 00000100:00100000:2.0:1769074959.566983:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54d1dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:69562:x1855007617076224:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 2585us (5599us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074959.567004:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13691 00000100:00100000:3.0:1769074959.570562:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.571349:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076352 00000100:00100000:2.0:1769074959.571359:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076352 00000100:00100000:2.0:1769074959.571409:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13692 00000100:00100000:2.0:1769074959.571419:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+27:69562:x1855007617076352:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074959.571438:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074959.571459:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46e00/0x7fe2323adfd56191 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879caaf expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.571537:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47800/0x7fe2323adfd56198 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.571554:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47800/0x7fe2323adfd56198 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:2.0:1769074959.571566:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47800/0x7fe2323adfd56198 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:2.0:1769074959.571614:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000bd0:0xb:0x0]: returning size 4096 00000004:00200000:2.0:1769074959.571619:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1769074959.571651:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46e00/0x7fe2323adfd56191 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caaf expref: 29 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.571666:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47800/0x7fe2323adfd56198 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caaf expref: 28 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.571676:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:2.0:1769074959.571678:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1769074959.571682:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e47800, rc 0) 00000100:00100000:2.0:1769074959.571751:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:69562:x1855007617076352:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 334us (1200us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074959.571759:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13692 00000100:00100000:1.0:1769074959.579134:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074959.579328:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076480 00000100:00100000:0.0:1769074959.579338:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076480 00000100:00100000:0.0:1769074959.579411:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13693 00000100:00100000:0.0:1769074959.579421:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:69562:x1855007617076480:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074959.579440:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074959.579461:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd5619f lrc: 2/0,0 mode: --/CR res: [0x240000405:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cab6 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.579531:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:0.0:1769074959.579547:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 2/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7fe2323adfd56159 expref: 6 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.579555:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 4/1,0 mode: --/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT1 00010000:00010000:0.0:1769074959.579589:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7fe2323adfd56159 expref: 6 pid: 35531 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1769074959.579602:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6512:1855007625297792:0@lo:104: 00000100:00100000:0.0:1769074959.579649:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074959.579701:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b54393e480 going to sleep for 11 seconds 00000100:00100000:3.0:1769074959.579840:0:6496:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297792 00000100:00100000:3.0:1769074959.579846:0:6496:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297792 00000100:00100000:3.0:1769074959.579874:0:6496:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:3.0:1769074959.579880:0:6496:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6512:x1855007625297792:12345-0@lo:104: 00010000:00010000:3.0:1769074959.579892:0:6496:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.579952:0:6496:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1769074959.579958:0:6496:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6512:x1855007625297792:12345-0@lo:104: Request processed in 79us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074959.579965:0:6496:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00010000:00010000:3.0:1769074959.579985:0:6501:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769074959.579992:0:6501:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.580000:0:6501:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b666b89400 already unused, calling callback (ffffffffc1908290) 00010000:00010000:3.0:1769074959.580005:0:6501:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1769074959.580013:0:6501:0:(mdt_handler.c:3944:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 4/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.580038:0:6501:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 2/0,0 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.580044:0:6501:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1769074959.580064:0:6501:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 1/0,0 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.580071:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff89b666b89400/0x7fe2323adfd56159 lrc: 0/0,0 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7fe2323adfd56160 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.580089:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074959.580091:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769074959.580093:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769074959.580100:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1769074959.580111:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625297920:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1769074959.580140:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074959.580169:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625297920 00000100:00100000:2.0:1769074959.580326:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6512:1855007625297792:0@lo:104: 00010000:00010000:2.0:1769074959.580346:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000100:00080000:1.0:1769074959.580490:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1769074959.580734:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625297920 00010000:00010000:3.0:1769074959.580758:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 3/0,0 mode: PW/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7fe2323adfd56159 expref: 7 pid: 35531 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1769074959.580791:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7244 00000100:00100000:3.0:1769074959.580798:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3650:x1855007625297920:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1769074959.580987:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1769074959.591150:0:7431:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968392 is committed 00010000:00010000:3.0:1769074959.591385:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135c00/0x7fe2323adfd56160 lrc: 0/0,0 mode: --/PW res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7fe2323adfd56159 expref: 6 pid: 35531 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.591406:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xb:0x0].0x0 (ffff89b67a317900) 00010000:00010000:3.0:1769074959.591411:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/1,0 mode: --/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074959.591420:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074959.591427:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 4/1,0 mode: PR/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT1 00010000:00010000:3.0:1769074959.591465:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769074959.591556:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1855007625297920:12345-0@lo:103:ldlm_bl.0 Request processed in 10756us (11413us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074959.591566:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7244 00000100:00100000:0.0:1769074959.591771:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3650:1855007625297920:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1769074959.592318:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00010000:00010000:2.0:1769074959.592329:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/1,0 mode: PR/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT1 00000004:00200000:2.0:1769074959.592389:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000405:0xb:0x0]: returning size 4096 00000004:00200000:2.0:1769074959.592395:0:6512:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1769074959.592441:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff89b549134200/0x7fe2323adfd5619f lrc: 0/0,0 mode: --/CR res: [0x240000405:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cab6 expref: 9 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.592463:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff89b549135200/0x7fe2323adfd561a6 lrc: 3/0,0 mode: PR/PR res: [0x240000405:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cab6 expref: 8 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.592472:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000405:0xb:0x0].0x0 (ffff89b67a317900) 00010000:00010000:2.0:1769074959.592474:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1769074959.592477:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135200, rc 0) 00000100:00100000:2.0:1769074959.592543:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:69562:x1855007617076480:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 13123us (13429us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074959.592551:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13693 00000100:00100000:3.0:1769074959.598378:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.598569:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076608 00000100:00100000:2.0:1769074959.598595:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076608 00000100:00100000:2.0:1769074959.598685:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13694 00000100:00100000:2.0:1769074959.598697:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:69562:x1855007617076608:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:2.0:1769074959.598718:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769074959.598740:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46e00/0x7fe2323adfd561ad lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cabd expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.598840:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598855:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598866:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598879:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598891:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598897:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598985:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598993:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.598999:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599005:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599012:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549e47000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.599016:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599024:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599039:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:2.0:1769074959.599043:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.599048:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599054:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47000/0x7fe2323adfd561b4 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599068:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599074:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599079:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599083:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549e46400 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:2.0:1769074959.599085:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599091:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599096:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x526f614a (ffff89b64effa400) 00010000:00010000:2.0:1769074959.599098:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.599101:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599106:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46400/0x7fe2323adfd561bb lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.599136:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46e00/0x7fe2323adfd561ad lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cabd expref: 29 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.599146:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:2.0:1769074959.599148:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769074959.599151:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e46e00/0x7fe2323adfd561ad lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cabd expref: 29 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1769074959.599160:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549e46e00, rc 0) 00000100:00100000:2.0:1769074959.599399:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:69562:x1855007617076608:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 704us (1033us total) trans 0 rc 301/301 00000100:00100000:2.0:1769074959.599409:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13694 00000100:00100000:3.0:1769074959.604542:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074959.604651:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076736 00000100:00100000:0.0:1769074959.604662:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076736 00000100:00100000:0.0:1769074959.604720:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13695 00000100:00100000:0.0:1769074959.604730:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+28:69562:x1855007617076736:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074959.604750:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074959.604771:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd561c2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cac4 expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.604883:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd561c9 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.604898:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd561c9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.604909:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd561c9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:0.0:1769074959.605053:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20d:0x0]: returning size 4096 00000004:00200000:0.0:1769074959.605058:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1769074959.605076:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd561c9 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.605100:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd561c2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cac4 expref: 30 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.605116:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135000/0x7fe2323adfd561c9 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cac4 expref: 29 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.605406:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20d:0x0].0x0 (ffff89b65436a800) 00010000:00010000:0.0:1769074959.605409:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769074959.605412:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135000, rc 0) 00000100:00100000:0.0:1769074959.605477:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c68000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:69562:x1855007617076736:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 747us (954us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074959.605736:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13695 00000100:00100000:1.0:1769074959.615420:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074959.626268:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617076992 00000100:00100000:0.0:1769074959.626284:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617076992 00000100:00100000:0.0:1769074959.626353:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13696 00000100:00100000:0.0:1769074959.626363:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+29:69562:x1855007617076992:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:0.0:1769074959.626382:0:8855:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769074959.626386:0:8855:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1769074959.626413:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549e47800/0x7fe2323adfd56198 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879caaf expref: 29 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.626438:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:0.0:1769074959.626443:0:8855:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769074959.626457:0:8855:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd561d0 lrc: 2/0,0 mode: --/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cacb expref: -99 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.626564:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 3/0,1 mode: --/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.626577:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.626586:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 3/0,1 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.626600:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd561de lrc: 3/0,1 mode: --/PW res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.626607:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd561de lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.626612:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd561de lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627083:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627094:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627101:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:0.0:1769074959.627110:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20e:0x0]: returning size 0 00010000:00010000:0.0:1769074959.627148:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627156:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 4/0,1 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627162:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 3/0,0 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627168:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627185:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074959.627189:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 4/0,0 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627197:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 5/0,0 mode: CW/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627206:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:0.0:1769074959.627209:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074959.627213:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 3/0,0 mode: --/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627219:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd561d7 lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000004:00010000:0.0:1769074959.627230:0:8855:0:(mdt_handler.c:4460:mdt_save_lock()) ### save lock request 000000000860b498 reply state 00000000079a6c0a transno 8589944146 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd561de lrc: 3/0,1 mode: PW/PW res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627240:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135400/0x7fe2323adfd561de lrc: 3/0,1 mode: TXN/TXN res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.627245:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x526f614a (ffff89b67a317000) 00010000:00010000:0.0:1769074959.627247:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1769074959.627250:0:8855:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1769074959.627278:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135800/0x7fe2323adfd561d0 lrc: 0/0,0 mode: --/CW res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 31 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.627291:0:8855:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 30 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1769074959.627298:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20e:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:0.0:1769074959.627300:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769074959.627302:0:8855:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549135e00, rc 0) 00000100:00100000:0.0:1769074959.627368:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:69562:x1855007617076992:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 1005us (11957us total) trans 8589944146 rc 0/0 00000100:00100000:0.0:1769074959.627375:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13696 00000100:00100000:1.0:1769074959.632425:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074959.632548:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617077248 00000100:00100000:3.0:1769074959.632561:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617077248 00000100:00100000:3.0:1769074959.632631:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13697 00000100:00100000:3.0:1769074959.632642:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:69562:x1855007617077248:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074959.632669:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074959.632693:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549278a00/0x7fe2323adfd561f3 lrc: 2/0,0 mode: --/CR res: [0x200000405:0x20e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cad2 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.632833:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.632848:0:6512:0:(ldlm_lock.c:681:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 32 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.632857:0:6512:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 4/0,1 mode: --/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.632890:0:6512:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 32 pid: 8855 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.632904:0:6512:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 32 pid: 8855 timeout: 2047 lvb_type: 0 00000100:00100000:3.0:1769074959.632917:0:6512:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625298048:192.168.204.60@tcp:104: 00000100:00100000:3.0:1769074959.632997:0:6512:0:(client.c:2690:ptlrpc_set_wait()) set ffff89b64bb8b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769074959.638529:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074959.638609:0:6512:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6512:1855007625298048:192.168.204.60@tcp:104: 00000100:00100000:2.0:1769074959.638642:0:6500:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617077376 00010000:00010000:3.0:1769074959.638644:0:6512:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000100:00100000:2.0:1769074959.638652:0:6500:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617077376 00010000:00010000:2.0:1769074959.638690:0:6500:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 33 pid: 8855 timeout: 2047 lvb_type: 0 00000100:00100000:2.0:1769074959.638731:0:6500:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 7245 00000100:00100000:2.0:1769074959.638743:0:6500:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:2409:x1855007617077376:12345-192.168.204.60@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769074959.638761:0:6500:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769074959.638768:0:6500:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 33 pid: 8855 timeout: 2047 lvb_type: 0 00010000:00010000:2.0:1769074959.638781:0:6500:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 33 pid: 8855 timeout: 2047 lvb_type: 0 00010000:00010000:2.0:1769074959.638802:0:6500:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549135e00/0x7fe2323adfd561e5 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cacb expref: 33 pid: 8855 timeout: 2047 lvb_type: 0 00010000:00010000:2.0:1769074959.638840:0:6500:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20e:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:2.0:1769074959.638845:0:6500:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: --/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.638855:0:6500:0:(ldlm_lock.c:709:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.638863:0:6500:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 4/0,1 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6512 initiator: MDT0 00010000:00010000:2.0:1769074959.638891:0:6500:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769074959.638951:0:6500:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:2409:x1855007617077376:12345-192.168.204.60@tcp:103:ldlm_bl.0 Request processed in 209us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074959.638960:0:6500:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 7245 00010000:00010000:3.0:1769074959.640006:0:6512:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.640019:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00000004:00200000:3.0:1769074959.640044:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20e:0x0]: returning size 0 00000004:00080000:3.0:1769074959.643426:0:6512:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2ca:0x0], index 0: 0 00000004:00080000:3.0:1769074959.643463:0:6512:0:(osp_object.c:1649:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x2cb:0x0], index 0: 0 00010000:00010000:3.0:1769074959.643613:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,1 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643624:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 2/0,0 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643631:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643636:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074959.643640:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 3/0,0 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643648:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 4/0,0 mode: EX/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643664:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20e:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:3.0:1769074959.643667:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769074959.643672:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 2/0,0 mode: --/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643682:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd561fa lrc: 0/0,0 mode: --/EX res: [0x200000405:0x20e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.643741:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549278a00/0x7fe2323adfd561f3 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x20e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cad2 expref: 32 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.643751:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20e:0x0].0x0 (ffff89b67a317e00) 00010000:00010000:3.0:1769074959.643753:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074959.643757:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549278a00/0x7fe2323adfd561f3 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x20e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cad2 expref: 32 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.643771:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549278a00, rc 0) 00000100:00100000:3.0:1769074959.643852:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5444a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:69562:x1855007617077248:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 11211us (11437us total) trans 8589944147 rc 301/301 00000100:00100000:3.0:1769074959.643861:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13697 00000100:00100000:0.0:1769074959.648308:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.648421:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617077632 00000100:00100000:2.0:1769074959.648433:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617077632 00000100:00100000:2.0:1769074959.648504:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4755 00000100:00100000:2.0:1769074959.648514:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+31:69562:x1855007617077632:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074959.648851:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:69562:x1855007617077632:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 291us (506us total) trans 8589944148 rc 0/0 00000100:00100000:2.0:1769074959.648860:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4755 00000100:00100000:1.0:1769074959.655571:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.655671:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617077888 00000100:00100000:2.0:1769074959.655683:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617077888 00000100:00100000:2.0:1769074959.655746:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4756 00000100:00100000:2.0:1769074959.655757:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+32:69562:x1855007617077888:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074959.656011:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:69562:x1855007617077888:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 255us (452us total) trans 8589944149 rc 0/0 00000100:00100000:2.0:1769074959.656019:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4756 00000100:00100000:1.0:1769074959.718263:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074959.718360:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617078144 00000100:00100000:3.0:1769074959.718370:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617078144 00000100:00100000:3.0:1769074959.718435:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13698 00000100:00100000:3.0:1769074959.718456:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:69563:x1855007617078144:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074959.718481:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074959.718503:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549278a00/0x7fe2323adfd56208 lrc: 2/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cad9 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.718588:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718602:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718615:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718627:0:6512:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718633:0:6512:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718639:0:6512:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718725:0:6512:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718732:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718736:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718742:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718747:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074959.718750:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718756:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718767:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:3.0:1769074959.718770:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074959.718775:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718781:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279600/0x7fe2323adfd5620f lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718793:0:6512:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718797:0:6512:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.718802:0:6512:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.719224:0:6512:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549279a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:3.0:1769074959.719228:0:6512:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.719233:0:6512:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.719241:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x526f614a (ffff89b67a317000) 00010000:00010000:3.0:1769074959.719244:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769074959.719247:0:6512:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.719253:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279a00/0x7fe2323adfd56216 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6512 initiator: MDT0 00010000:00010000:3.0:1769074959.719286:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549278a00/0x7fe2323adfd56208 lrc: 1/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cad9 expref: 34 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.719296:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:3.0:1769074959.719299:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074959.719303:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549278a00/0x7fe2323adfd56208 lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cad9 expref: 34 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.719312:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549278a00, rc 0) 00000100:00100000:3.0:1769074959.719380:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:69563:x1855007617078144:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 926us (1126us total) trans 0 rc 301/301 00000100:00100000:3.0:1769074959.719388:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13698 00000100:00100000:0.0:1769074959.724074:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:3.0:1769074959.724153:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617078400 00000100:00100000:3.0:1769074959.724165:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617078400 00000100:00100000:3.0:1769074959.724244:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13699 00000100:00100000:3.0:1769074959.724254:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+33:69563:x1855007617078400:12345-192.168.204.60@tcp:101:lfs.0 00010000:00010000:3.0:1769074959.724274:0:6512:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769074959.724295:0:6512:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279400/0x7fe2323adfd5621d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb7d6bf619879cae0 expref: -99 pid: 6512 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1769074959.724443:0:6512:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20d:0x0]: returning size 4096 00010000:00010000:3.0:1769074959.724545:0:6512:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279400/0x7fe2323adfd5621d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cae0 expref: 35 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.724558:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff89b6432e7e00) 00010000:00010000:3.0:1769074959.724561:0:6512:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769074959.724566:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549279400/0x7fe2323adfd5621d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.60@tcp remote: 0xb7d6bf619879cae0 expref: 35 pid: 6512 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1769074959.724579:0:6512:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff89b549279400, rc 0) 00000100:00100000:3.0:1769074959.724695:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:69563:x1855007617078400:12345-192.168.204.60@tcp:101:lfs.0 Request processed in 442us (631us total) trans 8589944150 rc 301/301 00000100:00100000:3.0:1769074959.724706:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13699 00000100:00100000:1.0:1769074959.730649:0:3642:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:0.0:1769074959.730759:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617078528 00000100:00100000:0.0:1769074959.730770:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617078528 00000100:00100000:0.0:1769074959.730880:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 13700 00000100:00100000:0.0:1769074959.730890:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:69563:x1855007617078528:12345-192.168.204.60@tcp:34:lfs.0 00010000:00010000:0.0:1769074959.730972:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 3/1,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731002:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731013:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 3/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731024:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 3/1,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731031:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731037:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731114:0:8855:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/1,0 mode: --/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731122:0:8855:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731128:0:8855:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00000004:00200000:0.0:1769074959.731167:0:8855:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000405:0x20e:0x0]: returning size 0 00000004:00200000:0.0:1769074959.731185:0:8855:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1769074959.731199:0:8855:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731213:0:8855:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731219:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 4/1,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731224:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 3/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731230:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731235:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134000 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074959.731240:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 4/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731248:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 5/0,0 mode: CR/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731260:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x0 (ffff89b65436a700) 00010000:00010000:0.0:1769074959.731263:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074959.731268:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 3/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731274:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134000/0x7fe2323adfd5622b lrc: 0/0,0 mode: --/CR res: [0x200000bd0:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731287:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 3/1,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731292:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 2/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731297:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731301:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134a00 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074959.731303:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 3/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731308:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 4/0,0 mode: PR/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731314:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd0:0xb:0x0].0x526f614a (ffff89b67a317000) 00010000:00010000:0.0:1769074959.731316:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074959.731318:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 2/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731323:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134a00/0x7fe2323adfd56232 lrc: 0/0,0 mode: --/PR res: [0x200000bd0:0xb:0x0].0x526f614a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731332:0:8855:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731337:0:8855:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731342:0:8855:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731346:0:8855:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff89b549134600 already unused, calling callback (ffffffffc18fb550) 00010000:00010000:0.0:1769074959.731348:0:8855:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731353:0:8855:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 4/0,0 mode: PR/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731359:0:8855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000405:0x20e:0x0].0x0 (ffff89b67a317a00) 00010000:00010000:0.0:1769074959.731362:0:8855:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769074959.731364:0:8855:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 2/0,0 mode: --/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00010000:00010000:0.0:1769074959.731369:0:8855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff89b549134600/0x7fe2323adfd56239 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x20e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8855 initiator: MDT0 00000100:00100000:0.0:1769074959.731722:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:69563:x1855007617078528:12345-192.168.204.60@tcp:34:lfs.0 Request processed in 830us (1079us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074959.731731:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 13700 00000100:00100000:1.0:1769074959.735617:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074959.735752:0:15947:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617078784 00000100:00100000:2.0:1769074959.735762:0:15947:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617078784 00000100:00100000:2.0:1769074959.735961:0:15947:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 4757 00000100:00100000:2.0:1769074959.735973:0:15947:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b6656c1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+34:69563:x1855007617078784:12345-192.168.204.60@tcp:35:lfs.0 00000100:00100000:2.0:1769074959.736378:0:15947:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b6656c1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:69563:x1855007617078784:12345-192.168.204.60@tcp:35:lfs.0 Request processed in 405us (770us total) trans 8589944151 rc 0/0 00000100:00100000:2.0:1769074959.736386:0:15947:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 4757 00010000:00010000:0.0:1769074959.837164:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769074960.864849:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769074961.314515:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625298176:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.314515:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625298560:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.322544:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074961.322610:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c847100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625298688:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.322706:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074961.322731:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c845180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625299200:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.323456:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074961.323493:0:3650:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625299712:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.323520:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074961.323619:0:32381:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298176 00000100:00100000:1.0:1769074961.323629:0:32381:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298176 00000100:00100000:1.0:1769074961.323692:0:32381:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:1.0:1769074961.323701:0:32381:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681596450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625298176:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324007:0:32381:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681596450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:025c5dec-f591-4c93-9b92-fac36a3bdc36+23:3650:x1855007625298176:12345-0@lo:400:kworker.0 Request processed in 305us (1469us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.324015:0:32381:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:1.0:1769074961.324056:0:28519:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298688 00000100:00100000:1.0:1769074961.324061:0:28519:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298688 00000100:00100000:1.0:1769074961.324111:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1663 00000100:00100000:1.0:1769074961.324118:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625298688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324167:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1855007625298688:12345-0@lo:400:kworker.0 Request processed in 49us (1460us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.324173:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1663 00000100:00100000:1.0:1769074961.324195:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299200 00000100:00100000:1.0:1769074961.324199:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299200 00000100:00100000:1.0:1769074961.324226:0:6512:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299712 00000100:00100000:1.0:1769074961.324230:0:6512:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299712 00000100:00100000:1.0:1769074961.324249:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2615 00000100:00100000:1.0:1769074961.324255:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625299200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324286:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1855007625299200:12345-0@lo:400:kworker.0 Request processed in 32us (831us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.324293:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2615 00000100:00100000:1.0:1769074961.324580:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2616 00000100:00100000:2.0:1769074961.324589:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074961.324592:0:6512:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625299712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.324627:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c846300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625299072:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324635:0:6512:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5476f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1855007625299712:12345-0@lo:400:kworker.0 Request processed in 46us (1112us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.324643:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2616 00000100:00100000:2.0:1769074961.324664:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1769074961.324687:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1769074961.324687:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625299584:0@lo:400:kworker.0 00000100:00080000:1.0:1769074961.324694:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1769074961.324706:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298560 00000100:00100000:1.0:1769074961.324711:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298560 00000100:00100000:2.0:1769074961.324717:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074961.324759:0:8396:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299584 00000100:00100000:2.0:1769074961.324762:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625298304:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324764:0:8396:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299584 00000100:00100000:1.0:1769074961.324789:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1664 00000100:00100000:1.0:1769074961.324795:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625298560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324864:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625298560:12345-0@lo:400:kworker.0 Request processed in 69us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.324870:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1664 00000100:00100000:1.0:1769074961.324882:0:8396:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1665 00000100:00100000:1.0:1769074961.324888:0:8396:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625299584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.324915:0:8396:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625299584:12345-0@lo:400:kworker.0 Request processed in 28us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.324920:0:8396:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1665 00000100:00100000:1.0:1769074961.324938:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299072 00000100:00100000:1.0:1769074961.324942:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299072 00000100:00100000:1.0:1769074961.324968:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2617 00000100:00100000:1.0:1769074961.324974:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625299072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.325001:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b547ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1855007625299072:12345-0@lo:400:kworker.0 Request processed in 28us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.325007:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2617 00000100:00080000:1.0:1769074961.325034:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769074961.325037:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074961.325039:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769074961.325339:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074961.325556:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625298816:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.325581:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074961.325600:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625299328:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.325621:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074961.325648:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1769074961.325655:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625298432:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.325676:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074961.325694:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b54c844380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625298944:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.325713:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074961.325729:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625299456:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.325751:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074961.327245:0:27843:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299456 00000100:00100000:2.0:1769074961.327267:0:27843:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299456 00000100:00100000:2.0:1769074961.327321:0:27843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1666 00000100:00100000:2.0:1769074961.327329:0:27843:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625299456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.327447:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:025c5dec-f591-4c93-9b92-fac36a3bdc36:3650:1855007625298176:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.327467:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3650:1855007625298688:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.327478:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c845180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3650:1855007625299200:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.327569:0:3650:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3650:1855007625299712:0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.327718:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298816 00000100:00100000:1.0:1769074961.327724:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298816 00000100:00100000:1.0:1769074961.327962:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298432 00000100:00100000:1.0:1769074961.327966:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298432 00000100:00100000:1.0:1769074961.327980:0:8855:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298944 00000100:00100000:1.0:1769074961.327982:0:8855:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298944 00000100:00100000:1.0:1769074961.327998:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2618 00000100:00100000:1.0:1769074961.328004:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625298816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.328081:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1855007625298816:12345-0@lo:400:kworker.0 Request processed in 77us (2498us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.328089:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2618 00000100:00100000:1.0:1769074961.328106:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2619 00000100:00100000:1.0:1769074961.328111:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625298432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.328143:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1855007625298432:12345-0@lo:400:kworker.0 Request processed in 31us (2465us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.328160:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2619 00000100:00100000:1.0:1769074961.328173:0:8855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2620 00000100:00100000:1.0:1769074961.328179:0:8855:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625298944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.328208:0:8855:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1855007625298944:12345-0@lo:400:kworker.0 Request processed in 29us (2493us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.328213:0:8855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2620 00000100:00100000:2.0:1769074961.328512:0:27843:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625299456:12345-0@lo:400:kworker.0 Request processed in 1183us (2759us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074961.328520:0:27843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1666 00000100:00100000:2.0:1769074961.328559:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c847480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625298560:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.328571:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c846300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3649:1855007625299072:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.328582:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625299584:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.328607:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625298304 00000100:00100000:2.0:1769074961.328611:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625298304 00000100:00100000:2.0:1769074961.328641:0:6517:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299328 00000100:00100000:2.0:1769074961.328644:0:6517:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299328 00000100:00100000:2.0:1769074961.328661:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2814 00000100:00100000:2.0:1769074961.328669:0:6517:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625298304:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769074961.328674:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2815 00000100:00100000:1.0:1769074961.328685:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625299328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.328705:0:6517:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625298304:12345-0@lo:400:kworker.0 Request processed in 37us (3365us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074961.328712:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2814 00000100:00100000:1.0:1769074961.328732:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625299328:12345-0@lo:400:kworker.0 Request processed in 48us (3109us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074961.328738:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2815 00000100:00100000:2.0:1769074961.328750:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3651:1855007625298432:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.328761:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3651:1855007625298944:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.330640:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625299456:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.330673:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b67b411880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625298304:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.330686:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3651:1855007625298816:0@lo:400:kworker.0 00000100:00100000:2.0:1769074961.330696:0:3651:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968392 gen 2 00000004:00080000:2.0:1769074961.330702:0:3651:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294968392 committed transno 4294968392 00000100:00100000:2.0:1769074961.330726:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b54c844000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625299328:0@lo:400:kworker.0 00000100:00080000:2.0:1769074961.332222:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1769074961.332231:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769074961.332235:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1769074961.885069:0:6501:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074962.374278:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074962.374338:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074962.374383:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074962.374411:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074962.374438:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.60@tcp (source: 12345-192.168.204.60@tcp) 00000100:00100000:2.0:1769074962.374466:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617079040 00000100:00100000:2.0:1769074962.374475:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617079040 00000100:00100000:2.0:1769074962.374527:0:10283:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617079168 00000100:00100000:2.0:1769074962.374530:0:10283:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617079168 00000100:00100000:2.0:1769074962.374553:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2621 00000100:00100000:2.0:1769074962.374562:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007617079040:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074962.374631:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549208380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+35:2409:x1855007617079040:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 70us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074962.374638:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2621 00000100:00100000:2.0:1769074962.374655:0:10283:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 2622 00000100:00100000:2.0:1769074962.374661:0:10283:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617079168:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074962.374686:0:10283:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b54920bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617079168:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 25us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1769074962.374692:0:10283:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 2622 00000100:00100000:2.0:1769074962.374711:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617079296 00000100:00100000:2.0:1769074962.374714:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617079296 00000100:00100000:2.0:1769074962.374739:0:36516:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617079424 00000100:00100000:2.0:1769074962.374742:0:36516:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617079424 00000100:00100000:2.0:1769074962.374763:0:36516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1667 00000100:00100000:2.0:1769074962.374769:0:36516:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617079296:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:2.0:1769074962.374796:0:36516:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b549209180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+8:2409:x1855007617079296:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 27us (458us total) trans 0 rc 0/0 00000100:00080000:1.0:1769074962.375287:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769074962.375293:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769074962.375296:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1769074962.375298:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1769074962.375382:0:36516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1667 00000100:00100000:0.0:1769074962.375453:0:32382:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007617078912 00000100:00100000:0.0:1769074962.375459:0:32382:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007617078912 00000100:00100000:0.0:1769074962.375496:0:32382:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 417 00000100:00100000:0.0:1769074962.375504:0:32382:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b681595050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007617078912:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:0.0:1769074962.375545:0:32382:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b681595050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a7835f93-1e5d-43a4-8110-91f072d9ae65+9:2409:x1855007617078912:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 42us (1161us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074962.375551:0:32382:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 417 00000100:00100000:3.0:1769074962.377185:0:28519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.60@tcp, seq: 1668 00000100:00100000:3.0:1769074962.377202:0:28519:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007617079424:12345-192.168.204.60@tcp:400:kworker.0 00000100:00100000:3.0:1769074962.377278:0:28519:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5437b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:249ac4ab-c06c-4166-99d5-c4c7e1cc2791+7:2409:x1855007617079424:12345-192.168.204.60@tcp:400:kworker.0 Request processed in 82us (2838us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074962.377286:0:28519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.60@tcp, seq: 1668 00010000:00010000:0.0:1769074962.909086:0:8413:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769074963.357956:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1769074963.357981:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625299840:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1769074963.358048:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769074963.358240:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1769074963.358253:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625299968:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1769074963.358288:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074963.358551:0:7791:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299840 00000100:00100000:1.0:1769074963.358560:0:7791:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299840 00000100:00100000:3.0:1769074963.358588:0:6518:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625299968 00000100:00100000:3.0:1769074963.358594:0:6518:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625299968 00000100:00100000:1.0:1769074963.358619:0:7791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2816 00000100:00100000:3.0:1769074963.358627:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2817 00000100:00100000:1.0:1769074963.358632:0:7791:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5462de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625299840:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1769074963.358637:0:6518:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625299968:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1769074963.358665:0:7791:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446113280 avail 1299873792 00000004:00000024:3.0:1769074963.358665:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1445343232 avail 1299103744 00000100:00100000:3.0:1769074963.358721:0:6518:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1855007625299968:12345-0@lo:41:osp-pre-0-1.0 Request processed in 86us (431us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074963.358724:0:7791:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5462de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1855007625299840:12345-0@lo:41:osp-pre-1-0.0 Request processed in 96us (675us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074963.358734:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2817 00000100:00100000:3.0:1769074963.358779:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b546c6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3651:1855007625299968:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1769074963.358846:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3652:1855007625299840:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1769074963.365464:0:7791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2816 00000100:00100000:3.0:1769074963.869209:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625300096:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769074963.869301:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074963.869376:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1769074963.869383:0:3649:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625300224:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074963.869408:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769074963.869483:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625300096 00000100:00100000:1.0:1769074963.869492:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625300096 00000100:00100000:1.0:1769074963.869548:0:8400:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625300224 00000100:00100000:1.0:1769074963.869552:0:8400:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625300224 00000100:00100000:1.0:1769074963.869577:0:8400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1316 00000100:00100000:1.0:1769074963.869586:0:8400:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625300096:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1769074963.869617:0:8400:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:1.0:1769074963.869668:0:8400:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1855007625300096:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1769074963.869676:0:8400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1316 00000100:00100000:0.0:1769074963.869691:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1317 00000100:00100000:0.0:1769074963.869702:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625300224:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1769074963.869721:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:0.0:1769074963.869758:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b546c6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1855007625300224:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074963.869765:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1317 00000100:00080000:3.0:1769074963.869874:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769074963.869878:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769074963.869912:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b547ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3649:1855007625300224:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1769074963.869929:0:3649:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5475dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3649:1855007625300096:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769074963.869998:0:3651:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625300352:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769074963.870029:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769074963.870219:0:8401:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625300352 00000100:00100000:3.0:1769074963.870224:0:8401:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625300352 00000100:00100000:3.0:1769074963.870277:0:8401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1318 00000100:00100000:3.0:1769074963.870285:0:8401:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625300352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769074963.870296:0:8401:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 484898112 pending 0 free 3914993664 avail 3683471360 00000100:00100000:3.0:1769074963.870338:0:8401:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1855007625300352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1769074963.870344:0:8401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1318 00000100:00100000:2.0:1769074963.870418:0:3651:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b6491edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3651:1855007625300352:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1769074963.870635:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1769074963.933036:0:6502:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769074964.381159:0:3652:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625300480:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1769074964.381231:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769074964.381468:0:16192:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855007625300480 00000100:00100000:0.0:1769074964.381478:0:16192:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855007625300480 00000100:00100000:0.0:1769074964.381548:0:16192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1319 00000100:00100000:0.0:1769074964.381558:0:16192:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625300480:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1769074964.381578:0:16192:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 59200832 pending 0 free 3915956224 avail 3684433920 00000100:00100000:0.0:1769074964.381626:0:16192:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff89b5475dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1855007625300480:12345-0@lo:13:osp-pre-1-1.0 Request processed in 70us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1769074964.381633:0:16192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1319 00000100:00100000:0.0:1769074964.381679:0:3652:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff89b5462dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3652:1855007625300480:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1769074964.381757:0:6509:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 Debug log: 70342 lines, 70342 kept, 0 dropped, 0 bad.